您好,我尝试通过CUDA_VISIBLE_DEVICES让程序在指定gpu运行,但是会出现以下报错:
Traceback (most recent call last):
File "/mnt/cephfs/ml_group/retrieval_qq_qa/T2Ranking-main/src/train_dual_encoder.py", line 307, in
main_cli()
File "/mnt/cephfs/ml_group/retrieval_qq_qa/T2Ranking-main/src/train_dual_encoder.py", line 280, in main_cli
torch.cuda.set_device(local_rank)
File "/root/miniconda3/envs/t2ranking/lib/python3.10/site-packages/torch/cuda/init.py", line 350, in set_device
torch._C._cuda_setDevice(device)
RuntimeError: CUDA error: invalid device ordinal
那么如果我想指定程序在第3,4,5号gpu上运行,该怎么做到呢
您好,我尝试通过CUDA_VISIBLE_DEVICES让程序在指定gpu运行,但是会出现以下报错: Traceback (most recent call last): File "/mnt/cephfs/ml_group/retrieval_qq_qa/T2Ranking-main/src/train_dual_encoder.py", line 307, in
main_cli()
File "/mnt/cephfs/ml_group/retrieval_qq_qa/T2Ranking-main/src/train_dual_encoder.py", line 280, in main_cli
torch.cuda.set_device(local_rank)
File "/root/miniconda3/envs/t2ranking/lib/python3.10/site-packages/torch/cuda/init.py", line 350, in set_device
torch._C._cuda_setDevice(device)
RuntimeError: CUDA error: invalid device ordinal
那么如果我想指定程序在第3,4,5号gpu上运行,该怎么做到呢