search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.78k stars 6.79k forks source link

cuDNN lib mismatch using mxnet 1.9.1 #21179

Open IzanCatalan opened 1 year ago

IzanCatalan commented 1 year ago

Description

When loading any mxnet (as mx in the picture) function, mx.nd.ones or mx.gpu(0) for example, I get a warning about cudnn mismatch after an error in cublas.

This cublas error is solved when instead of executing mx.gpu(0), i execute mx.gpu(). In both cases, GPU is detected. When the error did not appear, I noticed an underutilisation of the GPU.

I wonder if this underutilization is due to the mismatching. In that case, is there any solution to eliminate the warning without installing another version of cudnn or cuda?

Also, I execute onnxruntime-gp, but it looks like mxnet throws the warning.

Thanks!

Izan.

Environment

Mxnet-cu112 version 1.9.1 python version 3.8 cuda version 11.2 cudnn version 8.2.1 onnxruntime-gpu version 1.12

error gpu

github-actions[bot] commented 1 year ago

Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue. Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly. If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on contributing to MXNet and our development guides wiki.

jun297 commented 1 year ago

Is there any updates for this issue?