lucidrains / big-sleep

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun
MIT License
2.57k stars 304 forks source link

Multiple Phrases and "minmax" Training #40

Closed afiaka87 closed 3 years ago

afiaka87 commented 3 years ago

I added the ability to train on multiple prompts. Simply pass a \ delimiter ("\\" in python strings) to separate each phrase.

You can also send in phrases to penalize in the output. This can be surprisingly effective (or a total failure).

Train on Multiple Phrases

In this example we train on three phrases:

from big_sleep import Imagine

dream = Imagine(
    text = "an armchair in the form of pikachu\\an armchair imitating pikachu\\abstract",
    lr = 5e-2,
    save_every = 25,
    save_progress = True
)

dream()

Penalize certain prompts as well!

In this example we train on the three phrases from before,

and penalize the phrases:

dream = Imagine( text = "an armchair in the form of pikachu\an armchair imitating pikachu\abstract", text_min = "blur\zoom", ) dream()


### Basic experiment showing it works to a degree

This image is generated from the cell above. 

![pika_2](https://user-images.githubusercontent.com/3994972/109432745-528d0a00-79d2-11eb-967d-03f0bfcaa775.png)

What happens if we minimize the word "yellow"?

![armchair_pika_0](https://user-images.githubusercontent.com/3994972/109433102-38ecc200-79d4-11eb-98c7-fc1050def0ce.png)