Closed Qiancj0605 closed 2 years ago
Myelin is a deep learning compiler used by TensorRT, it did works like merge operations or perform higher-level optimization on the graph.
Thanks @zerollzeng .How to understand this part of the log information in screenshot?Is myelin optimized for all ops?
No, only some of the ops will be optimized by myelin. from the user's perspective, I don't think we need to care about myelin unless it reports errors.
For example, in order to reduce the kernel launch, I want to integrate several operators here, but I don't know if myelin has done it, so I'm a little confused。
you can use trtexec to convert the model with --verbose enabled, you will see the layer info(tactic, kernal, whether they are fused) in the log.
@Qiancj0605 , not sure if you are looking for reducing kernel launch latency? If so you still needs cuda graph. Because we might still launch many kernels for single myelin layer.
@ttyio YES, for reducing kernel launch latency。Can you provide cuda graph information, except this “https://developer.nvidia.com/blog/cuda-graphs/”?Thanks.
Hello @Qiancj0605 , I also find https://developer.nvidia.com/gtc/2020/video/cwe21914-vid, https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31312/ hope it helps.
Closing since no activity for more than 3 weeks, thanks!
@ttyio Hi,when I do model conversion(onnx->tensorrt), output part of the log。 What is Myelin, how to understand this part of the log information, and what optimizations have been made?Thanks