search

numenta / nupic.research

Experimental algorithms. Unsupported.
https://nupicresearch.readthedocs.io
GNU Affero General Public License v3.0
104 stars 60 forks source link

BERT Base 2x and 4x Wide Configs (recreated) #537

Closed lucasosouza closed 3 years ago

lucasosouza commented 3 years ago

This include configs for training BERT 2x and 4x wide.Respectively, these are

bert_sparse_trifecta_2x_100k
bert_sparse_trifecta_4x_100k

I've ran the LR range test for only a 2x model. A hardware error has prevented me from running for the 4x.

@lsouza You can run the 2x in the meantime via

run.sh bert_sparse_trifecta_2x_100k

This will use 4 p3dn instances.