afiaka87 / clip-guided-diffusion

A CLI tool/python module for generating images from text using guided diffusion and CLIP from OpenAI.
MIT License
457 stars 62 forks source link

image_prompts is mistakenly set to text prompts #13

Closed ghost closed 3 years ago

ghost commented 3 years ago

Hi, thanks for putting this project out there, I am having fun playing with it. I am using it from the command line. I tried to set the --image_prompts argument but it would fail at the beginning. For example, my command would be:

cgd --image_prompts='images/32K_HUHD_Mushroom.png' --skip_timesteps=500 --image_size 256 --prompt "8K HUHD Mushroom"

And I'd get the output:

Given initial image: 
Using:
===
CLIP guidance scale: 1000 
TV Scale: 100.0
Range scale: 50.0
Dropout: 0.0.
Number of cutouts: 48 number of cutouts.
0it [00:00, ?it/s]
Using device cuda. You can specify a device manually with `--device/-dev`
0it [00:04, ?it/s]
/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import fnmatch, glob, traceback, errno, sys, atexit, locale, imp, stat
Traceback (most recent call last):
  File "/usr/local/bin/cgd", line 33, in <module>
    sys.exit(load_entry_point('cgd-pytorch==0.1.5', 'console_scripts', 'cgd')())
  File "/home/milhouse/.local/lib/python3.9/site-packages/cgd/cgd.py", line 385, in main
    list(enumerate(tqdm(cgd_generator))) # iterate over generator
  File "/home/milhouse/.local/lib/python3.9/site-packages/tqdm/std.py", line 1127, in __iter__
    for obj in iterable:
  File "/home/milhouse/.local/lib/python3.9/site-packages/cgd/cgd.py", line 167, in clip_guided_diffusion
    image_prompt, batched_weight = encode_image_prompt(img, weight, image_size, num_cutouts=num_cutouts, clip_model_name=clip_model_name, device=device)
  File "/home/milhouse/.local/lib/python3.9/site-packages/cgd/cgd.py", line 97, in encode_image_prompt
    pil_img = Image.open(fetch(image)).convert('RGB')
  File "/home/milhouse/.local/lib/python3.9/site-packages/cgd/cgd.py", line 76, in fetch
    return open(url_or_path, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '8K HUHD Mushroom'

So, on this line here: https://github.com/afiaka87/clip-guided-diffusion/blob/b18753b3f49666fd7c2c824bb4ab24de8f397880/cgd/cgd.py#L354

I think you meant to write: image_prompts = args.image_prompts.split('|')

That seemed to fix the problem for me.

afiaka87 commented 3 years ago

Thanks that is indeed a bug; I'll be sure to patch it up.