search

nihaomiao / CVPR23_LFDM

The pytorch implementation of our CVPR 2023 paper "Conditional Image-to-Video Generation with Latent Flow Diffusion Models"
BSD 2-Clause "Simplified" License
432 stars 43 forks source link

Hand + Face of Human Pose #19

Closed p0mad closed 11 months ago

p0mad commented 11 months ago

Hi, Is it possible to generate a single character from the Pose for about 5 seconds?

I have a video of Pose ( openpose + hands + face) and i was wondering if it is possible to generate an output video withe the length of 5 seconds that has a consistent character/Avatar which plays Dance, .... from the controlled (pose) input?

I have a video of OpenPose+hands+face and i want to generate human like animation (No matter what, but just a consistent Character/Avatar) Sample Video

Thanks Best regards

nihaomiao commented 11 months ago

Hi, @p0mad, your task is more similar to pose-guided video generation. For controllable image generation (including pose), ControlNet is quite hot recently (paper, code). You may be also interested in my previous WACV23 work (paper, code), which uses the pose sequence of a driving video and some subject frames to generate a new video.

p0mad commented 11 months ago

@nihaomiao Thank you so much! The youtube Dance dataset is not much accurate on hands! while hands are the most important of my project! best regards

p0mad commented 11 months ago

@nihaomiao May i have your contact to talk about implementing an idea? Telegram, skype or linkedin

nihaomiao commented 11 months ago

Hi, @p0mad, thanks a lot for your interest! But I am busy recently and may not have time to do so. I think there are lots of work about your task. It should not be hard for you to find one, e.g., this CVPR 2022 paper.