search

lucasnewman / best-rq-pytorch

Implementation of BEST-RQ - a model for self-supervised learning of speech signals using a random projection quantizer, in Pytorch.
MIT License
84 stars 8 forks source link

Pre-training cost of BEST-RQ #6

Open sausage-333 opened 8 months ago

sausage-333 commented 8 months ago

Thanks for your great work.

I want to know how long does pre-train take for 0.3B models. Can you share your experience for the cost of pre-training BEST-RQ? (batch size, GPU you used, # of that GPU, training time, etc...)