search

m3hrdadfi / soxan

Wav2Vec for speech recognition, classification, and audio classification
Apache License 2.0
249 stars 37 forks source link

Running out of memory #4

Open connormeaton opened 3 years ago

connormeaton commented 3 years ago

Thanks for providing your great code. I have ran the wav2vec emotion classification of greek audio and it works great. However, I keep running into memory errors when changing the dataset.

I am trying to do emotion classification on the IEMOCAP dataset (https://sail.usc.edu/iemocap/), but it breaks due to running out of memory. I am changing nothing in your code except for the dataset (which fits into your pipeline well).

I made an SO post describing my issues. Do you have any ideas? I am trying to run a p2.8xlarge instance with 100 GiB mounted, so I can't imagine I don't have enough compute.

SO post: https://stackoverflow.com/questions/68624392/running-out-of-memory-with-pytorch

Thanks!

Jeiyoon commented 2 years ago

I've encountered same issue even though I have 8 gpus, with 11178MiB per gpu

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 10.92 GiB total capacity; 10.11 GiB already allocated; 7.38 MiB free; 10.12 GiB reserved in total by PyTorch)
vipen07 commented 1 year ago

I am also facing same issue , I have 4 gpus with same free available memory