search

Extraltodeus / noise_latent_perlinpinpin

This allows to create latent spaces filled with perlin-based noise that can actually be used by the samplers.
32 stars 7 forks source link

purple spots #4

Closed brycedrennan closed 8 months ago

brycedrennan commented 9 months ago

Hey this is a great idea. I'm trying to integrate it into imaginairy

I'm getting these spots in the sd15 images when I use the perlin noise. Is this something you've encountered?

I wonder if I need to use a different VAE

028619_1_ddim50_PS7 5_portrait_photo_of_a_woman_with_a_few_freckles _red_hair_ generated

If I change the clamp from result = torch.clamp(result, -5, 5) to result = torch.clamp(result, -4, 4) it significantly reduces the spots but I'm not sure what other negative consequences there might be. 028627_1_ddim50_PS7 5_portrait_photo_of_a_woman_with_a_few_freckles _red_hair_ generated

brycedrennan commented 9 months ago

result = torch.clamp(result, -3.9, 5) seemed to reduce the issue to negligible.

I tried the two different autoencoders from stability with both float16 and float32 but the spots remained.

Extraltodeus commented 8 months ago

Yeah I know. That's the reason I reworked things a lot. Unfortunately I have a feeling that the first earlier versions which were more prone to give this kind of issue were those bringing the most benefits to upscaling. I'm closing the issue.