search

romilbert / samformer

Official implementation of SAMformer, a transformer leveraging Sharpness-Aware Minimization and Channel-Wise Attention for Time Series Forecasting.
MIT License
130 stars 18 forks source link

Training with GPU reports error tensorflow and cudnn version mismatch #8

Closed lilyniu closed 3 months ago

lilyniu commented 5 months ago

error： 2024-06-12 16:19:09.672260: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:417] Loaded runtime CuDNN library: 8.3.2 but source was compiled with: 8.6.0. CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration. 2024-06-12 16:19:09.704946: E tensorflow/compiler/xla/status_macros.cc:57] INTERNAL: RET_CHECK failure (tensorflow/compiler/xla/service/gpu/gpu_compiler.cc:618) dnn != nullptr Begin stack trace tsl::CurrentStackTrace[abi:cxx11]()

Tried updating cudnn and still getting errors. (py38) sudo dpkg -l | grep libcudnn ii libcudnn8 8.6.0.163-1+cuda11.8 amd64 cuDNN runtime libraries ii libcudnn8-dev 8.6.0.163-1+cuda11.8 amd64 cuDNN development libraries and headers ii libcudnn8-samples 8.6.0.163-1+cuda11.8 amd64 cuDNN samples

romilbert commented 5 months ago

Hi,

Thank you for raising this issue. The error indicates a version mismatch between your current cuDNN library and TensorFlow. For TensorFlow, the version compiled must match the cuDNN runtime library version. You can check the compatibility for the correct versions here.

Romain