sczhou / ProPainter

[ICCV 2023] ProPainter: Improving Propagation and Transformer for Video Inpainting
https://shangchenzhou.com/projects/ProPainter/
Other
5.45k stars 645 forks source link

The result is not very good, do you want to train it? #44

Open wxllyf opened 11 months ago

wxllyf commented 11 months ago

sdasd0231003023923

0231003023848

user080975 commented 11 months ago

I also have the same issue when trying to remove people from videos, the removed areas come out as very pixelated or very blurred

Roowenliang commented 11 months ago

Is this an image or video?

wxllyf commented 11 months ago

Is this an image or video?

video

user080975 commented 11 months ago

This is the result I got from the walking video from the Davis dataset. I feel the result would be a lot better if somehow the inpainted areas aren't so blurry.

https://github.com/sczhou/ProPainter/assets/54892636/f3dd3898-2df8-4048-a937-6d4a936fbac6

https://github.com/sczhou/ProPainter/assets/54892636/9190f0ea-e795-4514-bf1c-0981dbbeec21

pdr0github commented 11 months ago

Results are better when you have clean background areas in other frames - either the people/objects have to move, or the camera has to move to reveal them . If the mask area obscures everything, and there are no clean textures behind the "object/people" in nearby frames - you cannot benefit from using other frames to inpaint

wxllyf commented 11 months ago

https://github.com/sczhou/ProPainter/assets/1106791/4260b1ea-573d-42bb-849f-10f3fac2ddbe

https://github.com/sczhou/ProPainter/assets/1106791/1cd48a7d-24cf-42f4-9e2d-6cbce39a1c55

Results are better when you have clean background areas in other frames - either the people/objects have to move, or the camera has to move to reveal them . If the mask area obscures everything, and there are no clean textures behind the "object/people" in nearby frames - you cannot benefit from using other frames to inpaint

Results are better when you have clean background areas in other frames - either the people/objects have to move, or the camera has to move to reveal them . If the mask area obscures everything, and there are no clean textures behind the "object/people" in nearby frames - you cannot benefit from using other frames to inpaint

Indeed, the current results in removing watermarks or text are decent, using only a pre-trained model.

user080975 commented 11 months ago

But if you look at the FGT demo video or try to run the exact same video using FGT, the results are a lot cleaner with the same Davis walking example for some reason. Is it because Pro Painter is based on SSTN that the output results are the same?

https://www.youtube.com/watch?v=BC32n-NncPs

2023-10-04 at 13 42 30
zaojue405 commented 11 months ago

can you tell me what tool you used to make mask?Do I need to draw the mask one frame by one frame?

wxllyf commented 11 months ago

can you tell me what tool you used to make mask?Do I need to draw the mask one frame by one frame?

sam-pt the project

a1207978731 commented 9 months ago

Hello, can you help me, I encountered a bug: FileNotFoundError: [Errno 2] No such file or directory: '/mnt/lustre/sczhou/VQGANs/CodeMOVI/experiments_model/recurrent_flow_completion_v5_train_flowcomp_v5/gen_760000.pth' how to get the weight file of recurrent flow completion.

zaojue405 commented 9 months ago

这是来自QQ邮箱的自动回复邮件。   您好,我已收到你的来信。我将尽快给您回复。