search

mailong25 / self-supervised-speech-recognition

speech to text with self-supervised learning based on wav2vec 2.0 framework
379 stars 114 forks source link

Unable to load multiple model #47

Closed vigneshgig closed 3 years ago

vigneshgig commented 3 years ago

Hi, I want to run multiple models. but after running one instance. i unable to run other instance. I am getting this error. ile "/media/administrator/hdd/self-supervised-speech-recognition/stt.py", line 187, in optimize_models model.cuda() File "/home/administrator/wav2vec_stt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 463, in cuda return self._apply(lambda t: t.cuda(device)) File "/home/administrator/wav2vec_stt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 359, in _apply module._apply(fn) File "/home/administrator/wav2vec_stt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 359, in _apply module._apply(fn) File "/home/administrator/wav2vec_stt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 359, in _apply module._apply(fn) [Previous line repeated 3 more times] File "/home/administrator/wav2vec_stt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 381, in _apply param_applied = fn(param) File "/home/administrator/wav2vec_stt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 463, in return self._apply(lambda t: t.cuda(device)) RuntimeError: CUDA error: all CUDA-capable devices are busy or unavailable please help me out. Gpu memory consuming for one instance is 2246 gb / 8 gb Thanks, @mailong25