Achieved real-time on RTX 3090 using TensorRT, reaching speeds of 30+ FPS.

[warmshao]

My implementation: https://github.com/warmshao/FasterLivePortrait New Features:

1. Achieved real-time performance of LivePortrait on RTX 3090 GPU using TensorRT, reaching speeds of 30+ FPS.
2. Implemented conversion of LivePortrait model to ONNX format, achieving inference speed of approximately 70ms/frame (~12 FPS) using onnxruntime-gpu on RTX 3090, facilitating cross-platform deployment.
3. Seamlessly integrated support for native Gradio app, delivering several times faster speed and enabling simultaneous inference on multiple faces. Sample results available at: PR #105
4. Refactored code structure to eliminate PyTorch dependency. All models now use ONNX or TensorRT for inference.

[FurkanGozukara]

amazing work congrats

[warmshao]

amazing work congrats

thanks! The speed is truly unbelievably fast. Perhaps it can be used for some interesting applications.

[galigaligo]

I still need to compile onnxrruntime gpu myself, which is a bit discouraging

[warmshao]

I still need to compile onnxrruntime gpu myself, which is a bit discouraging

The latest onnxruntime-gpu still doesn't support grid_sample cuda, so we need build it from source. But I will upload a docker image soon, stay tuned!

[juntaosun]

Very good, it runs at a steady 20FPS on RTX 3080 . 👍️

[juntaosun]

https://github.com/user-attachments/assets/d3ce17c8-c0d5-43c6-a3b3-f8f7d195c824

[shaoguowen]

FasterLivePortrait.mp4

wow, cool! Are you using tensorrt or onnx?

[warmshao]

Very good, it runs at a steady 20FPS on RTX 3080 . 👍️ cool

[warmshao]

hi guys, I have uploaded an docker image that supports docker running https://github.com/warmshao/FasterLivePortrait. Please try it out. I will provide integration packages for Windows and macOS that support one-click run. Stay tuned.

[vpckso]

Thanks, but not working

Could you share your Dockerfile so that I can build myself?

[warmshao]

Thanks, but not working

Could you share your Dockerfile so that I can build myself? You can try installing pycuda yourself: pip install pycuda. Actually, I follows the readme tutorial step by step to install in the container, then commit it, there's no Dockerfile.

[vpckso]

nvidia-smi also failed so the image cannot be used, I must build from scratch but I got a lot of compile error when follow the readme seems some libraries version not compatible

[shaoguowen]

nvidia-smi also failed so the image cannot be used, I must build from scratch but I got a lot of compile error when follow the readme seems some libraries version not compatible

pls refer this: https://github.com/warmshao/FasterLivePortrait/issues/8

[vpckso]

Thanks, it works after fix libcuda.so.1 and libnvidia-ml.so.1 also need to fix scripts/all_onnx2trt.sh to retinaface_det_static.onnx and face_2dpose_106_static.onnx

3060 with official pytorch, source/s6.jpg + driving/d0.mp3: real 0m16.065s user 0m19.367s sys 0m1.738s

compiled model can speed up around 3s

TensortRT: real 0m7.773s user 0m11.793s sys 0m11.129s

[warmshao]

Install-free, extract-and-play Windows package with TensorRT support now available! please refer FasterLivePortrait releases, Really fast and very convenient!!!

[falconwingz88]

will this work to a video target ?

[warmshao]

will this work to a video target ?

yes