search

continue-revolution / sd-webui-animatediff

AnimateDiff for AUTOMATIC1111 Stable Diffusion WebUI
Other
3.11k stars 258 forks source link

[Bug]: Cannot generate a GIF after choosing invalid frame in prompt travel #211

Closed ryuan closed 8 months ago

ryuan commented 1 year ago

Is there an existing issue for this?

Have you read FAQ on README?

What happened?

If you try to generate a prompt travel GIF by specifying an invalid frame number (ex., in a 16 frame GIF, specifying a trail prompt frame number of 16 when the largest valid frame number should be 15 given the 0-based index), you get the error IndexError: index 16 is out of bounds for dimension 0 with size 16.

Attempting to generate another GIF after correcting for this error (ex., reducing the tail prompt frame number down to 15 or attempting to generate a GIF without any prompt travel) produces a GIF of just random noise.

At the moment, the only fix is to completely exit+restart A1111 to reset everything. Reloading UI wasn't enough for me - it still produces an animated noise GIF.

Steps to reproduce the problem

  1. Attempt to generate a prompt travel GIF and specify a tail prompt with out-of-range frame number (ex., in a 16 frame GIF, specify 16).
  2. Press Generate and you should see an IndexError
  3. Now fix the tail prompt frame number (ex., to some number 15 or lower).
  4. Press Generate and you should see an animated GIF output of just random noise.
  5. Now reload the UI and repeat steps 3-4. You still get an animated GIF of random noise.
  6. Now exit+restart Automatic. You can now generate using the fixed prompt without issues.

What should have happened?

After getting the IndexError, one should be able to correct the invalid frame number, press "Generate", and get a valid output without having to close out and restart Automatic entirely.

Commit where the problem happens

webui: 1.6.0 extension: https://github.com/continue-revolution/sd-webui-animatediff/commit/552f50599783cc1814ea0309982be00ad6af677c

What browsers do you use to access the UI ?

Apple Safari

Command Line Arguments

No

Console logs

IndexError: index 16 is out of bounds for dimension 0 with size 16

Additional information

Please let me know if others are able to replicate this issue. I am on a Apple Silicon Mac, which always has its own particularities, so this issue could be isolated to M1/M2 Mac devices.

Also, if this issue is replicable, I'm happy to find a solution and submit a pull request. Let me know if you have any leads on why this issue is happening.

zappityzap commented 1 year ago

Looks like parse_prompt() in animatediff_prompt.py needs to check for invalid frame numbers.

continue-revolution commented 1 year ago

Not just this, but in general we should find a way to recover from any kind of errors. I have not implemented any error recovery strategy, and there might be no such strategy, but I’m not sure. Need further investigation.

GuruVirus commented 1 year ago

The readme needs to be updated for Prompt Travel example to show 0: and 7: not 8: Or change the math to start with frame 1, not 0.

continue-revolution commented 1 year ago

Why not 8? I assume that the context batch size is 16. I think this description is clear enough.

The second and third lines are for prompt interpolation, in format frame number: prompt. Your frame number should be in ascending order, smaller than the total Number of frames. The first frame is 0 index.

GuruVirus commented 1 year ago

Your example would have to be using context batch size 8 for this example to work. So frame 8 - 1 = frame index of 7.

A realistic example with context batch size 16 would be frame 16 listed as "15:" in the readme.