search

ahmetoner / whisper-asr-webservice

OpenAI Whisper ASR Webservice API
https://ahmetoner.github.io/whisper-asr-webservice
MIT License
1.99k stars 358 forks source link

Error loading heavy models #166

Closed anastasia-tskhay closed 10 months ago

anastasia-tskhay commented 10 months ago

I can't load models v2 and v3. The container freezes and nothing happens.

anastasia-tskhay commented 10 months ago

large v3 always stops in one place https://i.imgur.com/2bco96H.png

anastasia-tskhay commented 10 months ago

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 4.00 GiB total capacity; 2.30 GiB already allocated; 51.56 MiB free; 2.50 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF - also gives the same error. How much video memory do you need?

ayancey commented 10 months ago

large requires about 10 GB of VRAM. https://github.com/openai/whisper#available-models-and-languages