Open johndpope opened 2 months ago
+1
I'm doing currently very painful procedure because of this. Be able to faceswap on transparent png would be great feature.
ReActor face swap actually works... technically... if you take the transparent output from layerdiffuse and send it to Extras tab, then use ReActor.
However, it totally borks the image and makes it fully opaque
Btw I did make an issue over there, but it seems like ultimately ReActor would have to do some kind of magic to be compatible with images with alpha. layerdiffusion/sd-forge-layerdiffuse#50
Sorry to double post, but it seems like this could be a solution... I'll be trying it personally tomorrow when time permits
There're several issues we have in SD WebUI with all these things
Sorry to double post, but it seems like this could be a solution... I'll be trying it personally tomorrow when time permits ChatGPT says this works supposedly
But it's rather interesting solution if we could get alpha-png from the SD WebUI pipeline to postprocess it without the alpha-channel being cut Or we even could simply blend it by the face mask
Why using a transparent PNG before the face swap? Why not make a transparent PNG opaque, face swap and then remove the bg or use the transparency of the original to re-cut out the face swapped copy ... ? 🤷♂️
@Gourieff Or we even could simply blend it by the face mask
this would be great! If I understand it correctly, you would
This would be awesome and solve a lot of pain I'm currently doing in my pipeline to be even able to swap face to cropped characters :D
Also really good feature would be if you can return new faces as mask. I bet that would also help a lot of people.
Why using a transparent PNG before the face swap? Why not make a transparent PNG opaque, face swap and then remove the bg or use the transparency of the original to re-cut out the face swapped copy ... ? 🤷♂️
@Woisek One of the reason is because after you bring back bg, several other people are gonna be present and face indexes would be different. It cannot be automated without human intervention.
a hack from chatgpt - I asked it to remove this grid
RGB if item[0] == item[1] == item[2]: // eg. 3333333
problem is gray is a little off.
from PIL import Image
# Open the image file with the grid background
with Image.open("path_to_your_image.png") as img:
# Convert the image to RGBA if it's not already in that mode
if img.mode != 'RGBA':
img = img.convert('RGBA')
# Get the data of the image
datas = img.getdata()
newData = []
for item in datas:
# Check if the pixel is part of the grid (assumed to be shades of gray)
if item[0] == item[1] == item[2]:
# Make the grid transparent
newData.append((255, 255, 255, 0))
else:
newData.append(item)
# Update image data with new data
img.putdata(newData)
# Save the modified image with a transparent grid
img.save("path_to_your_new_image.png", "PNG")
@Woisek One of the reason is because after you bring back bg, several other people are gonna be present and face indexes would be different. It cannot be automated without human intervention.
Huh? What 'several other people are gonna be present' when you make the transparent image opaque with any color ... ??
Huh? What 'several other people are gonna be present' when you make the transparent image opaque with any color ... ??
Sry. Should have better describe it. My use case currently is stable-diffusio with inpainting person to custom backround. I have it automated via api but problem here is when the new background has multiple other characters and I don't know which id is the face I want to swap. I have the mask of person but when I feed masked image to reactor, it removes alpha channel. Because of this I have to use another layer that detect head a feeds only the head to reactor and that head is positioned back to composition.
tldr: if reactor wouldn't strip alpha channels life would be easier. I can imagine case where you have some people assets without background and you want them face swapped. Now it's impossible.
So what I said earlier that ChatGPT suggested, is working for my discord bot.
I just had to play around with it a bit.
layerdiffuse = img_payload['alwayson_scripts'].get('layerdiffuse', {})
if len(images) > 1 and layerdiffuse and layerdiffuse['args'][0]:
# Extract alpha channel from layerdiffuse output
img1 = images[1]
_, _, _, alpha = img1.split()
# Open ReActor output, convert to RGBA, apply alpha
img0 = Image.open(f'temp_img_0.png')
img0 = img0.convert('RGBA')
img0.putalpha(alpha)
# Save the image, update discord image payload
img0.save('temp_img_0.png')
images[0] = img0
Downsides:
if the layerdiffusion output has a 'mild cloudiness' to the background transparency layer, the result of this method here will have a faint checker pattern in those semitranslucent areas.
If the overall shape changes (head turned sideways), then the alpha won't align and it will look wonky
Ok guys With the quick commit https://github.com/Gourieff/sd-webui-reactor/commit/99b0f19553537a2f396a730455c9ea3a23623509 ReActor can work with RGBA via the Extras Tab (Single Image) - but only if SD WebUI doesn't cut Alpha channel when run postprocessing There's a quick fix to make SD WebUI allow RGBA for Extras: https://github.com/Gourieff/stable-diffusion-webui-forge/compare/main...Gourieff:stable-diffusion-webui-forge:extras--allow-png-rgba Fill free to test
@Gourieff Idk what im doing wrong but it still thinks it's RGB
ai-stable-diffusion-webui-1 | 10:22:13 - ReActor - DEBUG - We're here: process()
ai-stable-diffusion-webui-1 | 10:22:13 - ReActor - DEBUG - We're here: process()
ai-stable-diffusion-webui-1 | 10:22:13 - ReActor - STATUS - Working: source face index [0], target face index [0]
ai-stable-diffusion-webui-1 | 10:22:13 - ReActor - STATUS - image = <PIL.Image.Image image mode=RGB size=1080x1440 at 0x7F59341C2110>
ai-stable-diffusion-webui-1 | 10:22:13 - ReActor - STATUS - alpha = None
Using extra tab in webui. Tried even transparent png in target and source. Also could you do the fix also for API? Thx
webui version 1.8.0.
There's a quick fix to make SD WebUI allow RGBA for Extras: https://github.com/Gourieff/stable-diffusion-webui-forge/compare/main...Gourieff:stable-diffusion-webui-forge:extras--allow-png-rgba
@Dutch77 Did you apply this fix?
Also could you do the fix also for API?
Yes, this is in the plan as well
I'm gonna create seperate ticket. My problem is no longer on topic.
First, confirm
What happened?
I'm using with https://github.com/layerdiffusion/sd-forge-layerdiffuse + sd forge. when I use this plugin - it puts in a grid background instead of a transparent pixel.
Steps to reproduce the problem
install sd forge https://github.com/lllyasviel/stable-diffusion-webui-forge
add extension https://github.com/layerdiffusion/sd-forge-layerdiffuse
Sysinfo
ubuntu
Relevant console log
Additional information
.