search

sangmichaelxie / doremi

Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets
https://arxiv.org/abs/2305.10429
MIT License
286 stars 32 forks source link

Cuda version problem #27

Open RRaphaell opened 6 months ago

RRaphaell commented 6 months ago

I'm trying to run this script on aws sagemaker

cd doremi && bash scripts/setup_flash.sh

it throws an error:

The detected CUDA version (12.1) mismatches the version that was used to compile
      PyTorch (11.7). Please make sure to use the same CUDA versions.
Richard-Wth commented 2 months ago

May I ask if you have solved this problem? I face the same problem.

Richard-Wth commented 2 months ago

You can install a cuda 11.7 in /usr/local and add the following configuration to the bashrc file: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.7/lib64 export PATH=$PATH:/usr/local/cuda-11.7/bin export CUDA_HOME=/usr/local/cuda-11.7 Finally, run source bashrc. Then the compilation will succeed.