Text to image generation, Diffusion process in AI ..

Text to image generation by diffusion process in AI is an interesting area of Machine learning which builds models on vast amounts of data collected.

Deep fake AI

In Dream image generating AI app, I used the seeding words 'Image of a boy in a flowery meadow' and this is the result. (bottom left)

As different from deep faking apps, tech two years old, where people's portraits are inserted into existing images and videos, see my example, the latest technology is slightly interesting and advanced.

 

This worked using Generative Adversarial Networks (GAN)  having a generator and a discriminator. While the generator produces synthetic examples from random data, the discriminator component tries to distinguish between synthetic examples and real examples from a training dataset

 

Diffusion model on input Boy in a flowery meadow

AI has gone forward using Diffusion technology where using Contrastive Language Image Pretraining, diffusion system reconstructs data from noise, based on the word prompts. 

 

It is analogous to a master sculptor telling a novice where to chip a marble block to get a beautiful marble sculpture. 

 

On the right is an image generated using diffusion, where I have given the prompt words "George Easaw working in Alliance University Bangalore"

Diffusion model on prompt words of author at AU Bangalore

"At a high level, Diffusion models work by destroying training data by adding noise and then learn to recover the data by reversing this noising process. In Other words, Diffusion models can generate coherent images from noise. Diffusion models train by adding noise to images, which the model then learns how to remove" - scale.com

Diffusion model on prompt words of author at AU, Bangalore

The two high resolution images given to the left and right are generated from two different noise models. It is analogous to saying the two marble sculptures are taken from two different marble blocks excavated from two geographically different locations.

 

Click here for a practical guide to Diffusion Models.

Diffusion is applied these days in bio medicine to understand new treatments and to biochemistry to come out with new DNA sequences and molecules.

Diffusion models are generative models in the sense that they are trained to generate the same data output on which they are trained. How diffusion models work, click here ..

Sky is the limit, Diffusion is being used to compress images, generate videos, synthesise speech etc.

George