💡About the inference speed of onnx

juntaosun commented 3 months ago

# The part that causes the speed reduction ( warping_module )
out = self.live_portrait_wrapper.warp_decode(f_s, x_s, x_d_i_new)

After testing, warping.onnx is slower than the original .pth model. The main reasons are the following two points ( warping_module ):

(1) Exporting onnx does not support F.grid_sample 5D, and the non-native solution causes very slow running efficiency.
(2) The “encoder/decoder” in the model network “WarpingNetwork/DenseMotionNetwork/Hourglass” does not support quantization acceleration and is slower in onnx.

💡If you are planning to optimize the onnx model, please pay attention to the above two points.

x4080 commented 3 months ago

Hi @juntaosun have you tried this repo ? https://github.com/aihacker111/Efficient-Live-Portrait, it seems works faster on my apple silicon

aihacker111 commented 3 months ago

@juntaosun I’m tested 2 model spade generator and warp, but the reason is spade generator , it’s slower than warp , I’m checked the process onnx that some layer in this is take time to fall back CPU , please test my new model in repo , you can see it

juntaosun commented 3 months ago

solved

https://github.com/KwaiVGI/LivePortrait/issues/150#issue-2411686062

KwaiVGI / LivePortrait

💡About the inference speed of onnx #144