onnx->tensorrt

[Qiancj0605]

@ttyio Hi，when I do model conversion(onnx->tensorrt), output part of the log。 What is Myelin, how to understand this part of the log information, and what optimizations have been made?Thanks

[zerollzeng]

Myelin is a deep learning compiler used by TensorRT, it did works like merge operations or perform higher-level optimization on the graph.

[Qiancj0605]

Thanks @zerollzeng .How to understand this part of the log information in screenshot？Is myelin optimized for all ops?

[zerollzeng]

No, only some of the ops will be optimized by myelin. from the user's perspective, I don't think we need to care about myelin unless it reports errors.

[Qiancj0605]

For example, in order to reduce the kernel launch, I want to integrate several operators here, but I don't know if myelin has done it, so I'm a little confused。

[zerollzeng]

you can use trtexec to convert the model with --verbose enabled, you will see the layer info(tactic, kernal, whether they are fused) in the log.

[ttyio]

@Qiancj0605 , not sure if you are looking for reducing kernel launch latency? If so you still needs cuda graph. Because we might still launch many kernels for single myelin layer.

[Qiancj0605]

@ttyio YES, for reducing kernel launch latency。Can you provide cuda graph information, except this “https://developer.nvidia.com/blog/cuda-graphs/”？Thanks.

[ttyio]

Hello @Qiancj0605 , I also find https://developer.nvidia.com/gtc/2020/video/cwe21914-vid, https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31312/ hope it helps.

[ttyio]

Closing since no activity for more than 3 weeks, thanks!