What configuration were used to process these models? How much did it cost?

neuralmind-ai / portuguese-bert

Portuguese pre-trained BERT models

Other

792 stars 122 forks source link

What configuration were used to process these models? How much did it cost? #24

Closed rafaelsandroni closed 3 years ago

rafaelsandroni commented 3 years ago

Hi! I am curious about the resources requirements to obtain a model of this level.

What infrastructure configurations did you use to process these models, GPU, TPU, and RAM? What about the processing time? And if possible, how much did it cost to process all?

Thanks!!

fabiocapsouza commented 3 years ago

Hi @rafaelsandroni ,

The pretrainings take 4 days for the base model and 7 days for the large model using a TPUv3-8 device attached to a n1-standard-4 instance. This setup costs US$ 2,70/h using preemptible TPU or US$8,30/h using on-demand. Total cost would be about US$ 260 for base model and US$ 453 for large model. Data preprocessing steps can be run on common machines and so have negligible cost.