Open w-e-w opened 3 weeks ago
about the single image I know the cause and I have a way to fix it, but this method may brakes some extensions if you're interested you can see https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/more-extra-transparency-fix-again
basically the cause of the losing of transparency for P mode images is due the the gradio convertion the input image into RGBA
so when it gets to our script is already a RGBA image, the
when a P image gets converted into RGBA the transparency key
in removed as it is encoded into Alpha channel
up to this point is fine
next depending on the process module you use such as upscale upscale is for RGB only so the output of upscale is RGB not RGBA what has happened the transparency information is lost
it workd for Batch
because we are the ones doing the converting from P to RGB, and since we saved the tansparency key
befor converting and restore it after it's finished
transparency information is preserved
in this branch https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/more-extra-transparency-fix-again what I did is disable Gradios auto converting of image to RGBA, by disalbeing preprocess
, this way it retruns the raw image in base64 string format, doing so loud us more control on the converting process and allowing the preservation of the transparency key
the issue is as far as I'm aware disalbeing preprocess
applies for every component for that submit,
if there is currently any other extensions or modules that requires preprocessing (like if that moduls has a second image input)
they will also receive base64 as opposed to the image they're expecting, stuff wll break
so if there's a way of preserving the transparency key
after it's after it has been converted to RGBA
then things would work
another method is to set
https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/1c0a0c4c26f78c32095ebc7f8af82f5c04fca8c0/modules/ui_postprocessing.py#L15
from RGBA
bask to RGB
the change to RGB
was done in
yeah it's kind of ironic making RGBA possible for some modules actually breaks transparency for other image mode
yeah it's kind of ironic making RGBA possible for some modules actually breaks transparency for other image mode
Actually, here's an extra sample file to test this.
And here's the old sample file I am still using here as well:
With this, we have established that this commit: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15334
Is only enabling transparency for the reactor extension, not upscaling.
Identical to baseline.
Interestingly undoing the commit made it preserve transparency on one sample file but not the other when upscaling.
Identical to above.
One of the sample files still isn't transparent. Reactor successfully preserves transparency but fails to apply any changes to the face.
- With this, we have established that this commit: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15334 Is only enabling transparency for the reactor extension, not upscaling.
no we have not established that
what that PR dose it's enabling possibility of "Proper" RGBA support for single image input for ANY module as long as the module "itself" support transparency
upscaller depending on the upscaler method you use, and most dose not support transparency (Alpha Channel)
the reason you were able to get desirable transparency results is mostly a combination of luck
your image is a P mode image https://pillow.readthedocs.io/en/stable/handbook/concepts.html#modes
P mode: 8-bit pixels, mapped to any other mode using a color palette
which uses a different method of encoding transparency then RGBA as far as I'm weird image is essentially a grayscale image with a color palette information encoded alongside including the transparency of certain color palettes
the lucky part is if the color palettes transparency is save and re-apply back to the image you would get the transparency back as long as the processing module you're using does not shift the color
you can test it with different upscaling methods the more a particular message shifts the color the more likely that the transparency will be ruined
some of the test image you sent, image is basically composed of non don't transparent color parts and transparent background | img1 | img2 | img3 |
---|---|---|---|
test of img1 with different upscaling methods
Lanczos | R-ESRGAN 4x+ Anime6B | R-ESRGAN 4x+ | 16x-ESRGAN |
---|---|---|---|
transparency preserved with artifacts | image is complete gone leaving only transparent background | because of color shift you got blocks of black | transparency is completely gone |
if you are "lucky" and the color of the output image has the same color pallette as the input, it we then restore the info back transparency gets restored if you're unlucky then you might have results ranging from getting artifacts to completely ruined
the issue is not with that PR it's issue with lots of modules does not support transparency
in fact after taht PR it has better transparency support as long as all the modules you use in the processing pipeline support transparency
the reason that PR accidentally break transparency support for certain images you are using is because
the issue with gradio preprocess auto converting image into RGBA, during during conversion the the partial transparency metadata you have in your P mode image just converted to the more widely accepted and fully featured Alpha Channel this conversion is done before we have a chance to preserve the origin transparency metadata
and when later down the pipeling after going through upstairs that does not support transparency (Alpha) is lost and so when saving the image the transparency is lost
so the way I found to "fix" that is to take converting image into our own hands and not handled gradio but the orthodox method of doing so requires disableing preporcessing for the entire call the issues if we do so it has a potential of breaking extensions
what I would consider doing would be to patch gradio so that we have more contorl but I'm hesitant to do this now because I'm not sure when will we be upgrading to gradio4.x, because if we do so stuff like this sort of pathces will probably break
but the thing is I'm not entirely sure if doing all this is worth it as transparency is at best support finicky depending on your image contents and the upscaling method you're using
TypeError: cannot unpack non-iterable int object
and fail to save the imagepersonally I think it might be a good idea to just remove the transparency altogether because it could potentially ruin the image but I figure it might be a good idea to still preserve it on off chance that if it works for certain images
Yeah, including transparency is never going to be harmful, it's pretty hard to make an opaque image transparent, but the reverse is extremely easy, so it's never a bad thing.
But yeah, this PR solves the error it set out to solve perfectly.
Description
fix
when
enable_pnginfo
using extra "batch" theexisting_pnginfo
will also be restoredwhen converting a none RGB mode image to
RGB
someexisting_pnginfo
keys will also have to be converted to make it work this is the case with aP
mode image withTransparency
keythe issue is we currently extract the
existing_pnginfo
"before" we convert the image intoRGB
modeas suche the
existing_pnginfo
we restore later will have the old unconvertedexisting_pnginfo
fromP
modein the case of
P
-> toRGB
ifTransparency : 0
key exist, it will have to be converted in to a 3 value tuple(0, 0, 0)
this is why some users are seeing errorthe easiest solution I found is to convert the image to the to our desired mode before parseing the info with
read_info_from_image()
I have tested with the 4 formats of images generated by webui (png jpg webp avif) the info (the parts we wish to preserve) read after conversion seems to be intact
test images test images.zip test from #15421 #6534 2 p mode image with and without transparency key
before the PR the image with transparency key should faile to save using
extra batch
should be fixed after the PRnote I'm not an expert on image formats it's it is possible this modification can potentially messed up other things
Checklist: