williamyang1991 / VToonify

[SIGGRAPH Asia 2022] VToonify: Controllable High-Resolution Portrait Video Style Transfer
Other
3.52k stars 441 forks source link

Turning off Cropping in Video Toonification #52

Open ss8319 opened 1 year ago

ss8319 commented 1 year ago

Hi, I am wondering how to turn off automatic cropping in Video Toonification. What function and class is doing this?

I am working on processing some videos to be toonified using the Colab Notebook - PART II - Style Transfer with specialized VToonify-D model. How can I modify the code in this section to remove the automatic cropping. I did like to keep the output uncropped.

Thank you for the amazing work you are doing here.

ss8319 commented 1 year ago

@williamyang1991 I believe this is called 'rescaling' where the frames of the video is cropped to get just the face for VToonification.

I prefer to not rescale the video and keep it as it is. I am processing Sign Language videos and want to keep hand gestures. Is it possible? Or was VToonify build in such a way that it can only 'handle' faces.

Can I pad the frames so it doesn't get crop? Or use a different model?

williamyang1991 commented 1 year ago

How can I modify the code in this section to remove the automatic cropping. I did like to keep the output uncropped.

image

Or was VToonify build in such a way that it can only 'handle' faces.

VToonify is based on face toonification model (Toonify/DualStyleGAN). So basically, it will mainly change the face region in the image.

ss8319 commented 1 year ago

Thank you @williamyang1991

I am working in the Google Colab provided specificially, PART II - Style Transfer with specialized VToonify-D model.

Is there a direct way to add padding to Step 5: Video Toonification. Under this section, it shows that the input video needs to be rescaled. Screenshot (1062)

williamyang1991 commented 1 year ago

image