The object is not deleted, but is covered with black

[caslix]

Hello! Can you tell me what the problem is? As a result, the object was not deleted in the video, but became covered in black. The input data were videos and videos with a mask without an alpha channel.

Here is an example video: https://github.com/daniabib/ComfyUI_ProPainter_Nodes/assets/94461976/23da1451-ac7d-41ff-825b-c038b7dd7066

[daniabib]

Hi! It seems there is something wrong with how you are passing the mask to the model. Which node are you using to create or load the mask?

[caslix]

To download a mask, use the node "Load masks" . To create it - "Convert Image to Mask". "Red" is used as the channel, it stands by default. If you select "alpha", an error occurs:

An error occurred while executing ImageToMask: index 3 is outside dimension 3 with size 3

[daniabib]

It seems the problem is in how the Convert Image to Mask node is dealing with your mask video. Try to convert the mask video into png or jpg files and it should work.

[caslix]

But surely the LoadMask node implies downloading videos, not sequences?

[daniabib]

Sorry, indeed the node to load the masks in the example workflow is from VHS and uses video as input. I mean you can try to convert the video into an image sequence and load with the Load Video (Path) node to isolate the issue.

It is probably how your mask is being exported as a video before passing it to the ProPrainter node. You can also try to use the Invert Mask node before passing it to ProPainter.

[caslix]

Hello! The video was replaced, and the mask was replaced with sequences, but this did not give a result. The application of the "Invert Mask" node changed the output image. Now everything is black except the image being cut out.

Maybe it's still related to the use of the channel? (Red, Blue, Green, Alpha). Now it's Red, but if you set Alpha (which is more logical), an error is generated:

Error occurred when executing ImageToMask: index 3 is out of bounds for dimension 3 with size 3

[daniabib]

Do you mind sharing an example of the video and mask you are using so I can debug it? I tested with different video and mask sources and can't reproduce your results.

[caslix]

I use sequences from the official repository: https://github.com/sczhou/ProPainter/tree/main/inputs/object_removal

[daniabib]

Hey, sorry for the delay!

That's weird, It works fine for me with these images. Are you sure it's not on how you are converting these images into video? In the first screenshot, you're using a mp4 file, but the original images are png sequences. Have you tried to load the images directly with VHS's Load Video (Path) node? Does it still output black images?

[caslix]

Hello! Yes, I've tried that too. There was an error in the console, but in the end the task was completed

[daniabib]

Thank you, I will take a look at it.

Are you using Windows? If so, have you tried running ComfyUI with WSL2?

[caslix]

Hello! Sorry for the long answer. Yes, I use Windows for ComfyUI. I use WSL2 as needed. There are a lot of custom nodes in ComfyUI, so I won't be able to run it quickly on WSL2. I'll try to allocate a separate Portable Comfy for this as soon as I have time. Thanks!

[jokero3answer]

@daniabib 我的也是这样，输出黑色图像，help

[jokero3answer]

[jokero3answer]

I would like to try to combine this node with sam2

[daniabib]

Hi @jokero3answer! It is working fine for me using Kijai's SAM-2 implementation.

This issue is likely related to Windows, and I developed this node for Linux.

Can you give me more information about your system and Pytorch version? Also if you can include any error that shows in your ConfyUI console output would be helpful.

[jokero3answer]

Diagnostics-1723433125.log

[jokero3answer]

[jokero3answer]

@daniabib