The starting point
Diffusion models generate data by learning how to reverse a gradual corruption process. During training, noise is added to clean examples at different timesteps, and a neural network learns to predict the noise or another related target.
First principles
The forward process is a known noise schedule. The reverse process is learned. The model is not simply memorizing a finished image; it learns statistical structure that can guide iterative denoising from a noisy starting point.
| Concept | What it means | Why it matters |
|---|---|---|
| Forward process | Adds controlled noise over timesteps | Defines the corruption process |
| Denoising network | Predicts a noise-related target | Learns patterns in the training data |
| Sampling | Iteratively updates a noisy sample | Produces a generated candidate |
A practical example
A small example helps make the idea concrete. This snippet is illustrative; it assumes the surrounding data and setup are already in place.
noise = torch.randn_like(images)
noisy = scheduler.add_noise(images, noise, timesteps)
predicted = model(noisy, timesteps)
loss = torch.nn.functional.mse_loss(predicted, noise)What to take away
Keep data preprocessing, the noise schedule, the training objective, and the sampling procedure distinct when debugging. Generated quality depends on the complete pipeline, not one model setting.
The goal isn’t to memorize a recipe. It’s to understand the choices well enough to make them deliberately.