search

numenta / nupic.research

Experimental algorithms. Unsupported.
https://nupicresearch.readthedocs.io
GNU Affero General Public License v3.0
104 stars 60 forks source link

Better drop_last fix, updated results table (RES-2222) #521

Closed benja-matic closed 3 years ago

benja-matic commented 3 years ago

Replaced the old drop_last fix with a more durable solution, which overrides the predict and evaluate methods of the trainer. This fix covers the cases where those methods are called internally in HF code, not just by us.

Updated the readme results table: bert100k, bert_1mi, and sparse_80%_kd_onecycle_lr_rigl. The bert_1mi results are calculated using an extra 10 runs on wnli task.

There are some new experiments I'm playing with, like the "simple but hard to beat" baseline in finetuning.py. That stuff might still move around. Wanted to get this PR going for the results and the new fix.

mvacaporale commented 3 years ago

Have you tried running with tiny_bert_linear_lr_range_test? It would be good to know this can fully run without any issues as the lr-range tests validate every epoch.