search

Tencent / MimicMotion

High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance
https://tencent.github.io/MimicMotion/
Other
1.95k stars 166 forks source link

Replicate API Added! #23

Closed zsxkib closed 4 months ago

zsxkib commented 4 months ago

Hey @Tencent AI Lab team! 👋

I've created a Replicate demo for your amazing MimicMotion model. This allows users to easily run and experiment with MimicMotion through Replicate's platform.

Key features:

  1. Replicate Demo:

  2. Replicate API: Enables users to run the model directly from their code. Example usage:

    import replicate

output = replicate.run(
   "zsxkib/mimic-motion:2e7f79edf416dd7edd32dce85b7835be98eeb1d8eca2bebb2b8db8c996e5efad",
   input={
       "use_fp16": True,
       "chunk_size": 16,
       "resolution": 576,
       "motion_video": "https://replicate.delivery/pbxt/LD5c2cJou7MsS6J7KMBDfywggKAFCfsc2GUAlo67w4Z8aN30/pose1_trimmed_fixed.mp4",
       "sample_stride": 2,
       "frames_overlap": 6,
       "guidance_scale": 2,
       "noise_strength": 0,
       "denoising_steps": 25,
       "appearance_image": "https://replicate.delivery/pbxt/LD5c2GQlXTIlL1i3ZbVcCybtLlmF4XoPoTnbpCmt38MqMQiS/demo1.jpg",
       "output_frames_per_second": 15
   }
)
print(output)

I'm Sakib from Replicate, where we're working to make machine learning more accessible and reproducible. We were really impressed by MimicMotion and wanted to help make it easier for people to use and experiment with.

Adding the Replicate badge to your README would be great - it'll direct users to https://replicate.com/zsxkib/mimic-motion where they can try out the model instantly. This approach allows users to pay for their own compute, reducing the burden on your team.

To add the badge, you can use this markdown:

[![Replicate](https://replicate.com/zsxkib/mimic-motion/badge)](https://replicate.com/zsxkib/mimic-motion)

Let me know what you think! I'm excited to see how this can help more people experience the amazing capabilities of MimicMotion.

Keep up the fantastic work!

-Sakib

P.S. If you're interested in featuring MimicMotion on Replicate's homepage or blog, let me know. We'd love to showcase your innovation!

gujiaxi commented 4 months ago

@zsxkib Hi, a new MimicMotion model checkpoint supporting up to 72 frames of video generation has been released, greatly improving the generation quality. Could you please update the Replicate API with this new checkpoint? Thanks!

zsxkib commented 4 months ago

Hey @gujiaxi !

Thanks for the update. I’ll get on integrating the new MimicMotion checkpoint supporting 72 frames as soon as possible. Excited to see the improved generation quality!

Best, Sakib

zsxkib commented 4 months ago

Hey @gujiaxi and the @Tencent AI Lab team! 👋

Great news! I've integrated the new MimicMotion checkpoint supporting 72 frames as promised. The improved generation quality is truly impressive! 🎉

I've opened a new PR with all the updates here: https://github.com/Tencent/MimicMotion/pull/49

This new PR builds on our work here and includes some additional improvements to make the Replicate demo even better. It'd be awesome if you could check it out when you have a moment.

Thanks again for the fantastic update to MimicMotion!

Best, Sakib

ansonkao commented 4 months ago

Hey @zsxkib, this is great, thanks! Do you have any metrics from the 1000+ runs on replicate on how fast generation takes depending on which configurations are used? What are the fastest runs you've seen so far? Any insights on how to get MimicMotion to run faster?

Perhaps: