search

HarunoriKawano / BEST-RQ

Implementation of the paper "Self-supervised Learning with Random-projection Quantizer for Speech Recognition" in Pytorch.
https://arxiv.org/abs/2202.01855
Apache License 2.0
56 stars 2 forks source link

Pre-training cost of BEST-RQ #2

Open sausage-333 opened 8 months ago

sausage-333 commented 8 months ago

Thanks for your great work.

I want to know how long does pre-train take for 0.3B models. Can you share your experience for the cost of pre-training BEST-RQ? (batch size, GPU you used, # of that GPU, training time, etc...)

HarunoriKawano commented 8 months ago

Sorry. Unfortunately, I couldn't do pre-training of BEST-RQ because of resource.