search

amjltc295 / Free-Form-Video-Inpainting

Official Pytorch implementation of "Learnable Gated Temporal Shift Module for Deep Video Inpainting. Chang et al. BMVC 2019." and the FVI dataset in "Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN, Chang et al. ICCV 2019"
https://arxiv.org/abs/1907.01131
335 stars 52 forks source link

Config File to use for Model Trained on the Face Forensics Dataset #44

Open ashwath98 opened 3 years ago

ashwath98 commented 3 years ago

Hey guys, thanks for the clean code and open sourcing your work I'm looking into face inpainting techniques for some work in occlusion removal for faces, and I wanted to see if using your technique would be able to provide more temporal stability When I tried using the Face Forensics Weights (Forensics_L1_maskL1_vgg_style_1_6_1_10_allMasks_0311_092041_e200.pth ) with the normal testing script ( with some modification to load my custom data )

I am getting a shape mismatch error.

Am I expected to use a different model config file for the Face Foresnics model , If so which one?

Regards -Ashwath

kangzhiq commented 3 years ago

Thanks the authors for this great work!

Hi @ashwath98 , I am also running the Face model and encountered the same error as you. Basically, if you look at the error message, the mismatch parameters are in the deconv1/2 of UpSampleModule, with a factor of 2. So I checked how this model is initialized and found that the use_skip_connection option might be something related.

Briefly speaking, to solve your problem, you can change the False to True in this line https://github.com/amjltc295/Free-Form-Video-Inpainting/blob/19bfc6fbe626385be549ea2fc6c1ba7852e48e56/src/model/video_inpainting_model.py#L42

And normally the model should work fine.

SURABHI-GUPTA commented 2 years ago

Hi @kangzhiq, I am also running the Face model but getting errors. Did you try both LGTSM and TPGAN? Could you please help me with the inference code? If possible, could you share the steps to infer both models?