inference speed

[yyfyan]

Can you provide some compare about TensorRT and Pytorch at inference speed? thanks.

[chenwydj]

For our searched architecture (FasterSeg): TensorRT: 163.9 FPS Pytorch: 89.8 FPS

[yyfyan]

OK, thanks. TensorRT can process arbitrary pytorch model?

[chenwydj]

I think Pythorch supports more layer types and configurations. But for most common layer's, yes TensorRT does support.

[zhangyunming]

in the compare list ,does fast scnn or else use tensorrt?

[chenwydj]

• ENet, ICNet, CAS, Fast-SCNN: they did not release their code for latency.
• BiSeNet: They released their code. I can reproduce their FPS using TensorRT (but not with Pytorch).
• DF1-Seg-d8 (Li et al., 2019): yes they used TensorRT.

More works used TensorRT:

• SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization.
• Pelee: A Real-Time Object Detection System on Mobile Devices.
• IRLAS: Inverse Reinforcement Learning for Architecture Search.

[lxtGH]

@zhangyunming fast scnn use the TensorRT and its pytorch speed is only 70+ FPS

[chenwydj]

@lxtGH Where did you find this? I did not see this in Fast-SCNN paper.

[lxtGH]

@chenwydj I emailed to the author and also I test it in my implementation.

[chenwydj]

@lxtGH Thanks for sharing the info!

[lxtGH]

@chenwydj Could you share your speed on Camvid and BDD without using the Tensor-RT ??

[chenwydj]

@lxtGH Pytorch version FPS CamVid (720x960): 131.8 BDD (720x1280): 127.3