search

X-LANCE / UniCATS-CTX-txt2vec

[AAAI 2024] CTX-txt2vec, the acoustic model in UniCATS
https://cpdu.github.io/unicats
57 stars 8 forks source link

Hardware specs #5

Closed georgealexandruvlad closed 7 months ago

georgealexandruvlad commented 7 months ago

Hi!

First of all, congrats on the impressive work and thank you for taking the time to make it available on GitHub.

I have read the paper and I couldn't find any mention on the hardware setup for training. Any info about the hardware used for training and expected training times would be very much appreciated. I am currently trying a 4GPU setup with 16GB of VRAM per node and training with default configuration seems to raise OOM. So I was wondering what hardware configuration would be adequate.

Thanks!

danablend commented 7 months ago

24 GB vRAM setup worked fine for me. NVIDIA A10G, specifically. I could also run both training and inference just fine on my 4 GB laptop GPU.

Perhaps something else is causing you trouble, other than hardware?

georgealexandruvlad commented 7 months ago

Interesting. Then I'll have another look at what is actually causing the OOM now that the GPU memory constraint has been ruled out. Thanks!

cantabile-kwok commented 7 months ago

For your information, the training in the paper was done by V100 GPU with 32GB VRAM, so probably one needs to decrease the batch size if OOM occurs on GPUs with less VRAMs.