Closed rafaelsandroni closed 3 years ago
Hi @rafaelsandroni ,
The pretrainings take 4 days for the base model and 7 days for the large model using a TPUv3-8 device attached to a n1-standard-4 instance. This setup costs US$ 2,70/h using preemptible TPU or US$8,30/h using on-demand. Total cost would be about US$ 260 for base model and US$ 453 for large model. Data preprocessing steps can be run on common machines and so have negligible cost.
Hi! I am curious about the resources requirements to obtain a model of this level.
What infrastructure configurations did you use to process these models, GPU, TPU, and RAM? What about the processing time? And if possible, how much did it cost to process all?
Thanks!!