yownas / shift-attention

In stable diffusion, generate a sequence of images shifting attention in the prompt.
Other
164 stars 17 forks source link

TypeError: must be real number, not NoneType from clip.write_videofile() #11

Closed MoonRide303 closed 1 year ago

MoonRide303 commented 1 year ago

I just needed to see images for interpolated weight values in prompts (main and only use case for me), but it seems this extenesion enforces video generation, and fails at it - it was really disappointing to see all the images during rendering, and then, after multiple minutes of waiting, all gone (not even saved...), with that error message:

Traceback (most recent call last):
  File "D:\tools\Stable-Diffusion-web-UI\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "D:\tools\Stable-Diffusion-web-UI\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "D:\tools\Stable-Diffusion-web-UI\modules\txt2img.py", line 53, in txt2img
    processed = modules.scripts.scripts_txt2img.run(p, *args)
  File "D:\tools\Stable-Diffusion-web-UI\modules\scripts.py", line 407, in run
    processed = script.run(p, *script_args)
  File "D:\tools\Stable-Diffusion-web-UI\extensions\shift-attention\scripts\shift_attention.py", line 331, in run
    clip.write_videofile(os.path.join(shift_path, filename), verbose=False, logger=None)
  File "D:\anaconda3\lib\site-packages\decorator.py", line 232, in fun
    for i, extra in enumerate(extras):
  File "D:\anaconda3\lib\site-packages\moviepy\decorators.py", line 54, in requires_duration
    return f(clip, *a, **k)
  File "D:\anaconda3\lib\site-packages\decorator.py", line 232, in fun
    for i, extra in enumerate(extras):
  File "D:\anaconda3\lib\site-packages\moviepy\decorators.py", line 135, in use_clip_fps_by_default
    return f(clip, *new_a, **new_kw)
  File "D:\anaconda3\lib\site-packages\decorator.py", line 232, in fun
    for i, extra in enumerate(extras):
  File "D:\anaconda3\lib\site-packages\moviepy\decorators.py", line 22, in convert_masks_to_RGB
    return f(clip, *a, **k)
  File "D:\anaconda3\lib\site-packages\moviepy\video\VideoClip.py", line 300, in write_videofile
    ffmpeg_write_video(self, filename, fps, codec,
  File "D:\anaconda3\lib\site-packages\moviepy\video\io\ffmpeg_writer.py", line 213, in ffmpeg_write_video
    with FFMPEG_VideoWriter(filename, clip.size, fps, codec = codec,
  File "D:\anaconda3\lib\site-packages\moviepy\video\io\ffmpeg_writer.py", line 88, in __init__
    '-r', '%.02f' % fps,
TypeError: must be real number, not NoneType

Please at least allow disabling video generation (like on the screenshot in documentation, where "Save results as video" option is available, and can be turned off).

MoonRide303 commented 1 year ago

I looked at the code, and found the hint how to disable video generation:

save_video = video_fps != 0

so changing FPS to 0 is the usable workaround for this problem.

yownas commented 1 year ago

Hi, yes, the "Save results as video" checkbox was removed while I was cleaning up the ui because it started to get too cluttered, but I completely forgot to add that to the README. Sorry. :(

Even if the video generation failed, you should have copies of the images in (default) ../outputs/txt2img-images/shifts/

I'm still a bit curious about what went wrong, I haven't seen that error and not sure I know how to reproduce it. I'd love to fix it (if possible).

MoonRide303 commented 1 year ago

I don't remember the details, but it was pretty simple use case - I was using this extension to generate intermediate weights for negative embedding. It could be something like "(bad-image-v2-39000:0.8~1)", more or less.

From the code point of view I've included the full stack trace. Maybe you could catch the exception, and handle it in a way that would keep the static images in the UI, even in case video generation didn't work?

yownas commented 1 year ago

I added some error handling around the part that generates the video. Here: 4d90176830f7823d2a5fec0b3722f39f40fcc9fd

Still not sure what happen, it looks like it never got a value for "fps", but that should only be able to happen if the value is set to "nothing" and still larger than 0. ...either way. It should handle an empty value as if it was 0 now, and log an error instead of crashing if it can't create a video file. Doesn't really "solve" the issue, but handles it gracefully.