Black output from img2img

[pksebben]

Describe your environment

• GPU: m1 max
• VRAM: ?
• CPU arch: arm
• OS: macOS
• Python: Miniconda, as per installation instructions number 2
• Branch: main
• Commit: 8954171eea30557ea8f8b9bc0575c34c3630d64e

Describe the bug Black images in img2img generation

To Reproduce Steps to reproduce the behavior: Run img2img a bunch, wait for black output. See screencap for values used.

Expected behavior Slightly brighter images.

Screenshots

Additional context Problem is rare & erratic. Typically seen more when running with expensive parameters.

[skurovec]

I have maybe related issue, when generating using ddim, about 8 percent of images are black. When I recreate black images using same parameters, they are working. I have generated 300 images and 40 of them were black, same prompt, ddim, 512x768.

It was present on 1.14, but i think since 2.0 it is worse. But I hadn't this issue in rc-8.

Also M1 Max, 32GB.

[Any-Winter-4079]

I'm testing. In the meantime, could you write a command I could use to exactly mimic what you run? The screenshot has hidden values in variations. Also, please share your source image (if it's okay).

[Any-Winter-4079]

Here's a test with 48 images (512x512, k_euler_a, -s50, -f0.75, -C7.5) and no black images.

It may be more specific to your settings (e.g. a problem only when using Perlin noise, etc.) than a general problem in img2img

[Any-Winter-4079]

I'm doing a test much closer to your screenshot now. an anime girl -I 000003.640511255.png -n40 -Ak_euler_a -f0.55 -v0.6 -W576 -H576 -C2 --perlin 1 Update: no black images either.

I'll do a final test adding --threshold 8.1 Update. Same result.

I can't reproduce it from the CLI on 64GB RAM. @pksebben could you provide more info, e.g. source image and full list of the command options?

[Any-Winter-4079]

Finally could generate a black image

"an anime girl" -s 50 -S 450100192 -W 576 -H 576 -C 2.0 --perlin 1.0 --threshold 8.1 -I 000003.640511255.png -A k_euler_a -f 0.55 -V 928613172:0.6

Re-running the same command for the image it's no longer black.

[lstein]

Expected behavior Slightly brighter images.

Additional context Problem is rare & erratic. Typically seen more when running with expensive parameters.

You have Perlin noise turned way up (to 1.0). This will cause the dark brown images you are seeing, but shouldn't be causing the black ones. Turn Perlin down to 0.0 and try again.

[pksebben]

I've run this more, and found that at higher resolutions, the black output is more reliable. using this image as a source, with below settings... anything above 704px^2 seems to output black.

Perlin noise turned down as recommended.

I will admit that I am fumbling to a degree here, as I'm no expert in either machine learning nor image recognition. Certainly feels like this is bound to memory, though - as the occurrence of black boxes correlates to more memory-intensive procedures.

[pksebben]

addendum: @704px^2 black is output ~50% of the time.

[pksebben]

Also, problem seems correlated with heat. When I'm getting most of these black tiles, you could fry an egg on this laptop. I'm gonna let it cool for a minute and see if the problem goes away.

Might be limitation in the hardware.

[skurovec]

Please try latest master, there was some M1 specific code missing after merge.

[pksebben]

that seems to have fixed the problem. Thanks for the hard work, folks! This thing is an absolute beast.