I've used a fair amount of space in this blog to explain why I think AI might kill us all but there's a positive side to AI advancements as well.
For example, I've been using a image generating diffusion model for about a year now and it's never failed me. In fact, (I've used it for the cover photos for each of my blogposts.)
A little background on diffusion models:
diffusion models or diffusion probabilistic models, is a class of generative models used to generate realistic and high-quality images. One notable example of such a model is the Diffusion Probabilistic Models (DPM) framework.
At a high level, diffusion models leverage the concept of diffusion processes, which is a mathematical framework used to describe the spread of particles or information over time. In the context of image generation, the diffusion process refers to the gradual refinement of an image from a random noise source to the desired target image.
The key idea behind diffusion models is to model the relationship between an image and the noise applied at each step of the diffusion process. By iteratively applying a sequence of transformations, diffusion models can gradually reduce the noise and generate increasingly coherent images.
The specific implementation of diffusion models typically involves a series of steps known as diffusion steps. In each diffusion step, a noise source is applied to the current image, and a transformation operation is performed to update the image. The transformation operation could involve applying a neural network-based function or an invertible operation to manipulate the image pixels. These steps are repeated multiple times, gradually reducing the noise and refining the image.
To ensure that the generated images align with the desired target distribution (e.g., real-world images), diffusion models employ an optimization process known as denoising score matching. This process involves minimizing a loss function that encourages the generated image to match the statistics of the target image distribution. By iteratively optimizing this loss function, the model learns to generate images that exhibit realistic characteristics.
Diffusion models have gained popularity due to their ability to generate high-resolution and diverse images with fine-grained details. They can capture complex image structures, textures, and variations by iteratively refining the image representation through the diffusion steps.
Specifically, I've been using Disco Diffusion v5.7. Admittedly it doesn't produce the 1-to-1, prompt-to-final product results that some of the more high-profile image generators do so it might not be for everyone but I love it for who it is.
My favorite thing about it is the option to use an initial image which serves as a starting point or seed for the image generation process. The diffusion model gradually refines this initial image to generate a new image that aligns with the desired target distribution.
The process typically involves a series of diffusion steps, where each step involves applying a noise source and a transformation operation to update the image. However, instead of starting with random noise as the initial image, diffusion models can utilize an actual image as the starting point.
Here's a simplified explanation of how an initial image can be used in the diffusion model's image generation process:
Initialization: The diffusion model starts with an initial image, which could be a randomly generated image, a blank canvas, or even a real image from a dataset.
Diffusion Steps: The diffusion model performs a series of diffusion steps, typically iteratively. In each step, a noise source is applied to the current image, and a transformation operation is performed to update the image. The diffusion steps gradually reduce the noise and refine the image.
Refinement Process: The transformation operations within each diffusion step are designed to modify the image pixels based on the applied noise and the model's learned parameters. This refinement process helps the image move closer to the desired target distribution, which could be the distribution of real-world images.
Iterative Optimization: Throughout the diffusion steps, the model optimizes a loss function that encourages the generated image to match the target distribution. This optimization process helps the model learn the statistical properties and characteristics of real images, leading to the generation of visually coherent and realistic images.
By using an initial image as a starting point, diffusion models can leverage existing visual information, structure, or features present in the initial image. This can guide the image generation process and potentially result in images that retain some characteristics of the initial image while being transformed and refined through the diffusion steps.
Here are some links:
Disco Diffusion
Beginner's tutorial video
A more recent and in-depth tutorial
A tutorial specifically for using initial images