Closed babble2 closed 1 year ago
The pretraining using 10 A100s takes approximately 4 days with automatic mixed precision (torch.amp). The total batch size is 40, i.e., the batch size of each A100 is 4.
Hello, now I have a problem, can you help me, I made some changes on your model and use the same configuration repeated experiments many times but get different results, how can solve it? By the way,When you experiment with 10 A100s, “Batch” is set to 40?
------------------ Original ------------------ From: @.>; Date: Sun, Jun 11, 2023 09:01 PM To: @.>; Cc: @.>; @.>; Subject: Re: [shannanyinxiang/SPTS] pretrain time (Issue #13)
The pretraining using 10 A100s takes approximately 4 days with automatic mixed precision (torch.amp). The total batch size is 40, i.e., the batch size of each A100 is 4.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
How long did it take you to pretrain the existing A100,and how big batch of A100, thx!