Closed mvacaporale closed 3 years ago
This include configs for training BERT 2x and 4x wide.Respectively, these are
bert_sparse_trifecta_2x_100k bert_sparse_trifecta_4x_100k
I've ran the LR range test for only a 2x model. A hardware error has prevented me from running for the 4x.
@lsouza You can run the 2x in the meantime via
run.sh bert_sparse_trifecta_2x_100k
This will use 4 p3dn instances. Slack me if you have any questions.
This include configs for training BERT 2x and 4x wide.Respectively, these are
I've ran the LR range test for only a 2x model. A hardware error has prevented me from running for the 4x.
@lsouza You can run the 2x in the meantime via
This will use 4 p3dn instances. Slack me if you have any questions.