search

Paperspace / ml-in-a-box

Machine learning tool-set for Paperspace VMs
54 stars 23 forks source link

New MLiaB PyTorch is compiled for CUDA 11.6 #10

Closed nmb-paperspace closed 7 months ago

nmb-paperspace commented 1 year ago

The new MLiaB is CUDA 11.7 but the PyTorch 1.12.1 is compiled for CUDA 11.6.

This is an artifact of the time between when the MLiaB Docker was created and the current PyTorch 1.13.1 which defaults to 11.7.

Code that has strict checking for this may fail, e.g.,

Exception: Installed CUDA version 11.7 does not match the version torch was compiled with 11.6, unable to compile cuda/cpp extensions without a matching cuda version.

In this case it was solved by upgrading torch to 1.13.1, but we are noting the issue here as a generic case.

nmb-paperspace commented 7 months ago

The latest MLiaB is on CUDA toolkit 12.1.1, driver 535.129.03, and the PyTorch matches, so this is resolved