Suggestion: Image > mask node

[anxcon]

Like loadimage/loadimagemask nodes has mask output, but that requires loading an image when prior processing has an image, we don't need to load, but need a path to create mask output

[fambaa]

I am not a programmer, but I don't think thats hard to do. Maybe a simple node with an image input and a mask output would do it. And maybe have an option to choose the alpha channel, black, white, alpha and disable.

That way we can use filternodes to manipulate a generation to create a mask in the workflow for further use. I think that would be incredibly useful down the line.

I already have a few ideas on how it could be used, and I am just waiting for the day someone makes nodes like this.

[abbail]

I want to understand the need here. You are wanting a node that takes in an image and outputs a mask directly, something like this mockup?

[fambaa]

Yes, but not just RGB, also black and white. Its not about being able to pull a mask from loading a picture, rather creating a mask from a sampler inside the workflow.

[abbail]

When you say sampler, do you mean you want to, for example, take the LATENT output of a KSampler and turn it into a MASK or the IMAGE output of a VAEDecode?

[fambaa]

From Decode. What I am basically trying to do is using a depth map preprocessor to create an image, then run that through image filters to "eliminate" the depth data to make it purely black and white so it can be used as a pixel perfect mask to mask out foreground or background. Or if the preprocessor lets you control the depth, even select an mask a specific depth if that makes sense.

Look here: https://civitai.com/models/22651/comfyui-workflow-remove-background

Found this by accident, but it gave me the idea of doing it differently, more precise and on the original generated picture without having to generate again and have it change because of the weights changing.

[abbail]

Ok, so something like this maybe?

I'll upload it to my repo tomorrow and you can try it out assuming I don't find any killer bugs.

[fambaa]

That looks amazing! It seems to work exactly like I thought it could! I think it will be a useful addition. Thank you! Unfortunately the setlatentnoisemask node doesn't seem to care about the mask, I think it wants not a pixel mask but a latent type mask, which is weird. The cutting out and replacing the background with the depth map generated mask isnt as good as my other method, but I can probably figure out how to fit it better, but its progress.

PS: I took the liberty of grabbing the code to test, I couldn't wait, when I refine my workflow with it and figure things out I will link to your repo and credit you obviously ;)

[abbail]

Glad it worked for you. I didn't post it yet since I knew there were some breaking changes I'd need to make. For one I added the ability to choose a channel when doing the MonochromaticMask. If you get latest you'll need to tweak it slightly. If you find any bugs feel free to make a PR or let me know and I'll see if I can easily fix them:

• ImageToMask
• MonochromaticClip
• InvertMask

Here is an example that incorporates most of the features, I fit as much useful info on the screen at once. The top result is what you were asking for more or less, the bottom is its inverse:

[fambaa]

I am doing this with it, the mask is one part of the puzzle, although I think Comfy understands any BW latent image as a mask btw. Thats how the first version sortof worked. I fed it the BW depthmap as latent input, and it mostly only changed the image there. But not always. Thats where that green conditioning part came in with the weird depth control picture.

Anyways, this is different, I use the mask to render the same picture again, you get from a normal generation. Then using inpaint to add a background.

Thanks for your help with the nodes. Much appreciated. Maybe you can figure out what kind of mask the SetLatentNoisemask needs.

Seems to be a different kind of mask, also not really sure what it does, maybe useful in a way to manipulate latent data without a sampler? idk.

[abbail]

The default way an image mask is loaded is:

1. If you import an image with LoadImage and it has an alpha channel, it will use it as the mask. If there is no alpha channel, an entirely unmasked MASK is outputted.
2. If you import an image with LoadImageMask you must choose a channel and it will apply the mask on the channel you choose unless you choose a channel that doesn't exist and then it will output an entirely unmasked MASK.

Maybe those depth maps have an alpha channel? If it works somehow other than that I'm interested in understanding how. I just read the code and couldn't figure out how it might be applying a mask based on black or white besides using a channel.

Do you have an example workflow where you are using SetLatentNoisemask? I've not used that feature before but maybe I can make some sense of it if I can see the issue firsthand.

[fambaa]

Nah, I have not tried that yet. I looked into nodes.py and it seems it expects latent? The other thing is that if you use a controlnet like depth as condition input for a sampler it kinda acts like a mask. Thats what seems to be happening in my first workflow, otherwise I wouldn't have gotten that black background.

And we can't really use alpha, because the AI doesn't generate an alpha channel. Again, I am not even sure what latentnoisemask could be used for, I have also not dabbled with that yet.

I'll keep you updated ;)

[abbail]

I went ahead and uploaded these and a few more that I have made to CivitAI. I also updated the MonochromaticClip node to have a lot more features.

[robinjhuang]

@anxcon Do you still need this?