search

NVIDIA / ChatRTX

A developer reference project for creating Retrieval Augmented Generation (RAG) chatbots on Windows using TensorRT-LLM
Other
2.76k stars 339 forks source link

What version of Cuda is needed? #54

Closed leandroMsouza closed 6 months ago

leandroMsouza commented 6 months ago

I'm trying to run with the engine (llama_float16_tp1_rank0.engine)

but I'm getting this error

1

anujj commented 6 months ago

Have do done the installation thought the installer released by nvidia ? or you are doing manual setup ?

leandroMsouza commented 6 months ago

by Nvidia Installer

anujj commented 6 months ago

Could you please translate the error in English for better understanding. We have not seen any error from our side related to this this dll form our side

thiagojramos commented 6 months ago

Could you please translate the error in English for better understanding. We have not seen any error from our side related to this this dll form our side

I stumbled upon this by chance, but to not leave it unanswered, here's what the error message says in English:

Failed to locate the entry point for the 
procedure 
?destroyTensorDescriptorExOps@cudaNN@@YA?AW4cudnnStatus_t@@PEAPEAUcudnnTensorStruct@@@Z in the 
dynamic link library C:\Program Files\Python310\Lib\site-packages\nvidia\cudnn\bin\cudnn_adv_infer64_8.dll.

And to answer the question:

it's probably an error with the CUDA/CuDNN library installation or a version mismatch between the CUDA, CuDNN, and PyTorch/TensorFlow

leandroMsouza commented 6 months ago

Thanks, I'll re-install everything