Generations with Batch Size > 1 often error out

[obfuscode]

Issue: When Neutral Prompt extension is enabled, even if you're not using any of the prompt AND_* trigger words, changing the batch size > 1 often results in an error where the first image generates fine but the remaining images display glitched visuals and eventually throw a VAE error.

Reproducible: Yes, if you don't get the JS error reported in #64.

Prompt:

a photo of a bicycle
Steps: 20, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 526409519, Size: 512x512, Model hash: 338b85bc4f, Model: juggernaut_reborn, CFG Rescale phi: 0, Version: v1.7.0

(note: It does not seem to matter what Checkpoint you use)

Error Message:

*** Error completing request 20/20 [00:01<00:00,  9.64it/s] 
*** Arguments: ('task(u7pfcmw5h72gu64)', 'a photo of a bicycle', '', [], 20, 'DPM++ 2M Karras', 1, 2, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000002398DE1FF70>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\processing.py", line 734, in process_images
        res = process_images_inner(p)
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\processing.py", line 875, in process_images_inner
        x_samples_ddim = decode_latent_batch(p.sd_model, samples_ddim, target_device=devices.cpu, check_for_nans=True)
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\processing.py", line 603, in decode_latent_batch
        raise e
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\processing.py", line 600, in decode_latent_batch
        devices.test_for_nans(sample, "vae")
      File "C:\Users\hello\pinokio\api\automatic1111.git\app\modules\devices.py", line 150, in test_for_nans
        raise NansException(message)
    modules.devices.NansException: A tensor with all NaNs was produced in VAE. Use --disable-nan-check commandline argument to disable this check.

https://github.com/ljleb/sd-webui-neutral-prompt/assets/2160089/251edb97-002a-4f0d-a92e-b19fc339bcbe

[miasik]

The same here. During rendering it paints correctly only the first image and all others fade to black At the last step I see painted first image and all black other and than they disappear and the error appear: NansException: A tensor with all NaNs was produced in VAE. This could be because there's not enough precision to represent the picture. Try adding --no-half-vae commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.

[ljleb]

Thanks, I probably messed up something in the last update. I remember testing batch_size > 1 and even pushed a commit for it, but maybe I missed something.

Will push a fix shortly.

[obfuscode]

It did break the batch so even before it bombs out with the VAE error, all the batch images are identical.

I also noticed that it runs even if there isn't any of the AND_* keywords and I think it does change the outcome of the image (but adding the keyword to the prompt changes the image anyway so it's hard to test).

Can you add a check for the existence of the AND_* keywords and if not present just return out of the create_sampler_hijack()?

It looks like the only check in there is for an unsupported sampler and otherwise it runs no matter what the prompt.

~I can add this as a separate issue if you want?~ I'll make it a separate issue as I have another request related to it :)

[ljleb]

The last commit on main about delta-space should render prompts that only use 0 or more normal AND keywords exactly the same, whether the extension is active or not. You can test this quickly using the enabled checkbox in the settings. Is it possible you are not using the latest commit?

I'll reverify this to be sure.

[obfuscode]

I am using version 185638f2 | 2024-01-28 16:22:51 although I just ran the updater and see the new commits option 🤦so I applied and restarted and now it says current and the version is still 185638f2 | 2024-01-28 16:22:51 so I have no idea lol

[ljleb]

https://github.com/ljleb/sd-webui-neutral-prompt/commit/185638f2 is the last commit. I guess this is a problem in the latest commit then, I'll try to reproduce shortly.

[ljleb]

I just pushed a fix, it should not be a problem anymore. I was using the flattened prompt index directly instead of looking into the cond indices list for the actual unet batch index.

[ljleb]

I tested the small difference issue and, at least on my side, I cannot reproduce it in the latest commit:

edit: wow github is buggy. I can't save space by collapsing the images. Oh well.

png info:

a cute cub
Negative prompt: mediocre ugly abstract art by xynon-bad-11k-2
Steps: 24, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 1577655194, Size: 512x512, Model hash: e16273c410, Model: tmndMix_tmndMixVIIBakedvae, VAE hash: e169f60bfa, VAE: kl-f8-anime2.vae.pt, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", FreeU Schedule: "0.0, 1.0, 0.0", FreeU Version: 2, TI hashes: "xynon-bad-11k-2: 796c84e2a27c", Version: v1.7.0-RC-15-gcf2772fa

generation with neutral prompt enabled:

generation without neutral prompt:

[obfuscode]

I had to run unexpectedly but I'll test this in a bit and close it if it's all good. Thank you!

[obfuscode]

Seems to work great! Thanks for the quick fix!