Pretraining time on 16 A100 GPUs?

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

https://arxiv.org/abs/2305.07922

BSD 3-Clause "New" or "Revised" License

2.74k stars 401 forks source link

Open cherryheaven opened 4 months ago

cherryheaven commented 4 months ago

Could you give the running time for unimodal and bimodal pretraining? Thank you.