search

hyintell / BLOOM-fine-tuning

Finetune BLOOM
40 stars 10 forks source link

CUDA out of memory #3

Open wakeb98 opened 1 year ago

wakeb98 commented 1 year ago

i got CUDA out of memory and my resources are :

+-----------------------------------------------------------------------------+ | NVIDIA-SMI 528.89 Driver Version: 528.89 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla M60 TCC | 00000000:00:1B.0 Off | 0 | | N/A 33C P0 38W / 150W | 7600MiB / 7680MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 1 Tesla M60 TCC | 00000000:00:1C.0 Off | 0 | | N/A 34C P8 15W / 150W | 9MiB / 7680MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 2 Tesla M60 TCC | 00000000:00:1D.0 Off | 0 | | N/A 28C P8 15W / 150W | 9MiB / 7680MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 3 Tesla M60 TCC | 00000000:00:1E.0 Off | 0 | | N/A 33C P8 14W / 150W | 9MiB / 7680MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 6124 C ...akeb\anaconda3\python.exe 7586MiB | +-----------------------------------------------------------------------------+