search

zibojia / COCOCO

Video-Inpaint-Anything: This is the inference code for our paper CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibility.
https://zibojia.github.io
272 stars 5 forks source link

About Video inpainting results #10

Open Mr-Kyrie opened 2 weeks ago

Mr-Kyrie commented 2 weeks ago

Hi I tested a short 10 frame videos. but the inpainting result is poor. anything wrong about my testing? input resolution is 432 × 240 and without any promot. 1728458028708

zibojia commented 2 weeks ago

We have noticed this problem, this is mainly due to that the model that you used is not the best model. We will released the best model which can remove objects without prompt after this paper is accepted.

Mr-Kyrie commented 2 weeks ago

Do you mean The CoCoCo checkpoints are not the best?

zibojia commented 2 weeks ago

Yes, the model on onedrive is the weak version.

Mr-Kyrie commented 2 weeks ago

OK, Thanks for replying. I have another question, is the video length used for inference only applicable to shorter videos, about 10-20 frames? I have predicted the video using 96-frame video will cause dimension mismatch problems, just like RuntimeError: The size of tensor a (11088) must match the size of tensor b (10000) at non-singleton dimension 1

zhangp365 commented 1 week ago

OK, Thanks for replying. I have another question, is the video length used for inference only applicable to shorter videos, about 10-20 frames? I have predicted the video using 96-frame video will cause dimension mismatch problems, just like RuntimeError: The size of tensor a (11088) must match the size of tensor b (10000) at non-singleton dimension 1

I have the same problem, and I modify the max_len to 20000, and it passes. image I can't find any inpainting effect. image when running, It is often OOM on my two 4090 with 24 vram.

mychina75 commented 4 days ago

We have noticed this problem, this is mainly due to that the model that you used is not the best model. We will released the best model which can remove objects without prompt after this paper is accepted.

sorry to say, why could you release the best model only after paper is accpeted? Show the better result during paper under review could help the accepetance, maybe.

zibojia commented 4 days ago

We have noticed this problem, this is mainly due to that the model that you used is not the best model. We will released the best model which can remove objects without prompt after this paper is accepted.

sorry to say, why could you release the best model only after paper is accpeted? Show the better result during paper under review could help the accepetance, maybe.

Yes, we will release the best model after its acceptance. The model uses same arch but with more pleasing removal effect and won't generate the shutterstock watermark.