search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.68k stars 2.12k forks source link

Error Code 1: Myelin (Compiled against cuBLASLt 11.5.1.0 but running against cuBLASLt 11.4.2.0.) #2580

Closed jsjs0827 closed 1 year ago

jsjs0827 commented 1 year ago

how to solve?

zerollzeng commented 1 year ago

Can you provide the full log? I have a vague memory that this won't lead to the program crash.

oujiafan commented 1 year ago

We used TensorRT in Triton server, and got similar error.

http | [W] [TRT] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.4.0 http | [E] [TRT] 1: [raiiMyelinGraph.h::RAIIMyelinGraph::24] Error Code 1: Myelin (Compiled against cuBLASLt 11.5.2.0 but running against cuBLASLt 11.5.1.0.)

zerollzeng commented 1 year ago

We used TensorRT in Triton server, and got similar error.

I would suggest seeking help from Triton first. this looks like an env issue.

ttyio commented 1 year ago

closing since no activity for more than 3 weeks, please reopen if you still have question, thanks!

faithfulnguyen commented 4 months ago

you got that message because when you convert to TRT, the liblibcublas.so.11.4.2.0 was loaded, but when you do an inference, the lib cuBLASLt 11.5.1.0 was loaded instead oflibcublas.so.11.4.2.0, you need to install cuda version that has cuBLASLt 11.5.1.0 on your env used for convert model, you could looking at this link to find the cuda version for cuBLASLt 11.5.1.0 cuBLASLt 11.5.1.0 belongs to cuda 11.3.1 Use this code to check what libs loaded during conversion and inference.

def get_loaded_libraries():
    import os
    import psutil
    p = psutil.Process(os.getpid())
    for lib in p.memory_maps():
        print(lib.path)