search

NVIDIA / cuda-python

CUDA Python Low-level Bindings
https://nvidia.github.io/cuda-python/
Other
809 stars 63 forks source link

RuntimeError: Function "cuMemAllocAsync" not found #23

Closed vincentxiaotang closed 1 year ago

vincentxiaotang commented 2 years ago

When I ran the example file TensorFlowToTensorRT-NHWC.py, it occurs：Traceback (most recent call last): File "TensorFlowToTensorRT-NHWC.py", line 161, in _, inputD0 = cudart.cudaMallocAsync(inputH0.nbytes, stream) File "cuda/cudart.pyx", line 16938, in cuda.cudart.cudaMallocAsync File "cuda/ccudart.pyx", line 1210, in cuda.ccudart.cudaMallocAsync File "cuda/_cuda/ccuda.pyx", line 4970, in cuda._cuda.ccuda._cuMemAllocAsync RuntimeError: Function "cuMemAllocAsync" not found.

I'm using the NVIDIA NGC nvcr.io/nvidia/tensorflow:21.12-tf1-py3, the detailed environment are as follows: GeForce RTX 2080 Ti，Driver Version: 455.23.05，nvcr.io/nvidia/tensorflow:21.12-tf1-py3， Package Version

absl-py 1.0.0 appdirs 1.4.4 argon2-cffi 21.1.0 asgiref 3.4.1 astor 0.8.1 astunparse 1.6.3 attrs 21.2.0 audioread 2.1.9 backcall 0.2.0 bleach 4.1.0 cachetools 4.2.4 certifi 2021.10.8 cffi 1.15.0 charset-normalizer 2.0.8 click 8.0.3 cloudpickle 2.0.0 cmake-setuptools 0.1.3 cuda-python 11.7.0 cudf 21.10.0a0+345.ge05bd4bf3c cugraph 21.10.0a0+102.gab401cad cuml 21.10.0a0+116.gdc14361ba cupy-cuda114 9.3.0 cupy-cuda115 9.6.0 cycler 0.11.0 Cython 0.29.24 dask 2021.9.1 dask-cuda 21.10.0 dask-cudf 21.10.0a0+345.ge05bd4bf3c dask-glm 0.2.0 dask-ml 1.9.0 debugpy 1.5.1 decorator 5.1.0 defusedxml 0.7.1 distributed 2021.9.1 Django 3.2.6 entrypoints 0.3 fastavro 1.4.4 fastrlock 0.8 filelock 3.4.0 flatbuffers 1.12 fsspec 2021.7.0 future 0.18.2 gast 0.3.3 google-pasta 0.2.0 graphsurgeon 0.4.5 grpcio 1.42.0 gunicorn 20.1.0 h11 0.12.0 h5py 2.10.0 HeapDict 1.0.1 horovod 0.22.1 httptools 0.2.0 huggingface-hub 0.0.12 idna 3.3 importlib-metadata 4.8.2 importlib-resources 5.4.0 iniconfig 1.1.1 ipykernel 6.6.0 ipython 7.30.0 ipython-genutils 0.2.0 jedi 0.18.1 Jinja2 3.0.3 joblib 1.1.0 json5 0.9.6 jsonschema 4.2.1 jupyter-client 7.1.0 jupyter-core 4.9.1 jupyter-tensorboard 0.2.0 jupyterlab 2.3.2 jupyterlab-pygments 0.1.2 jupyterlab-server 1.2.0 jupytext 1.13.2 Keras-Applications 1.0.8 Keras-Preprocessing 1.0.5 kiwisolver 1.3.2 librosa 0.9.1 llvmlite 0.36.0 locket 0.2.1 Markdown 3.3.6 markdown-it-py 1.1.0 MarkupSafe 2.0.1 matplotlib 3.4.3 matplotlib-inline 0.1.3 mdit-py-plugins 0.2.8 mistune 0.8.4 mock 3.0.5 msgpack 1.0.3 multipledispatch 0.6.0 nbclient 0.5.9 nbconvert 6.3.0 nbformat 5.1.3 nest-asyncio 1.5.4 networkx 2.6.3 nltk 3.6.4 notebook 6.4.3 numba 0.53.1 numpy 1.22.4 nvidia-dali-cuda110 1.8.0 nvidia-dali-tf-plugin-cuda110 1.8.0 nvidia-dlprofviewer 1.8.0 nvidia-pyindex 1.0.9 nvtx 0.2.3 onnx 1.11.0 onnxruntime-gpu 1.11.1 opencv-python 4.5.5.64 opt-einsum 3.3.0 packaging 21.3 pandas 1.2.5 pandocfilters 1.5.0 parso 0.8.3 partd 1.2.0 pexpect 4.7.0 pickleshare 0.7.5 Pillow 8.4.0 pip 21.3.1 pluggy 1.0.0 polygraphy 0.33.0 pooch 1.6.0 portpicker 1.3.1 prometheus-client 0.12.0 prompt-toolkit 3.0.23 protobuf 3.19.1 psutil 5.7.0 ptyprocess 0.7.0 py 1.11.0 pyarrow 5.0.0 pycparser 2.21 Pygments 2.10.0 pynvml 11.4.1 pyparsing 3.0.6 pypi-kenlm 0.1.20210121 pyrsistent 0.18.0 pytest 6.2.5 python-dateutil 2.8.2 python-dotenv 0.19.2 pytz 2021.3 PyYAML 6.0 pyzmq 22.3.0 regex 2021.11.10 requests 2.26.0 resampy 0.2.2 rmm 21.10.0a0+42.gae27a57 sacremoses 0.0.46 scikit-learn 0.24.0 scipy 1.4.1 Send2Trash 1.8.0 setuptools 59.4.0 six 1.16.0 sortedcontainers 2.4.0 SoundFile 0.10.3.post1 sqlparse 0.4.2 tblib 1.7.0 tensorboard 1.15.0 tensorflow 1.15.5+nv tensorflow-estimator 1.15.1 tensorrt 8.2.1.8 termcolor 1.1.0 terminado 0.12.1 testpath 0.5.0 tf2onnx 1.10.1 threadpoolctl 3.0.0 tokenizers 0.10.3 toml 0.10.2 toolz 0.11.2 tornado 6.1 tqdm 4.62.3 traitlets 5.1.1 transformers 4.9.1 treelite 2.1.0 treelite-runtime 2.1.0 typing_extensions 4.0.1 ucx-py 0.21.0a0+37.gbfa0450 uff 0.6.9 urllib3 1.26.7 uvicorn 0.15.0 uvloop 0.16.0 watchgod 0.7 wcwidth 0.2.5 webencodings 0.5.1 websockets 10.1 Werkzeug 2.0.2 wheel 0.37.0 whitenoise 5.3.0 wrapt 1.13.3 xgboost 1.4.2 zict 2.0.0 zipp 3.6.0

vzhurba01 commented 2 years ago

Driver Version: 455.23.05

Driver 455 is associated with CUDA Toolkit 11.1 (CTK 11.1). However cudaMallocAsync/cuMemAllocAsync was introduced in CUDA Toolkit 11.2.

Getting the latest (or at minimum 11.2) Toolkit + driver should resolve this error.

vincentxiaotang commented 2 years ago

Driver Version: 455.23.05

Driver 455 is associated with CUDA Toolkit 11.1 (CTK 11.1). However cudaMallocAsync/cuMemAllocAsync was introduced in CUDA Toolkit 11.2.

Getting the latest (or at minimum 11.2) Toolkit + driver should resolve this error.

thank you, i'll have a try.