The Cerebros package is an ultra-precise Neural Architecture Search (NAS) / AutoML that is intended to much more closely mimic biological neurons than conventional neural network architecture strategies.
Kind of issue: [bug | feature-request-or-enhancement | Process Change | Support request]: Final fine tuning to the merge candidate: 233e8823201e8ea197e04c3caa9fabe6f773b907
Additional context
Last commit looks like our cold start performance is at parity with GPT2's pre-trained performance. It did run for 2 1/2 hours. Goals here:
Fine tune the model search to a constrained range of at or near optimal values.
Reduce the number of sub-trials and epochs.
Maybe, see if we can get away with increasing the sequence length further and if it is worth it in terms of embedding performance.
Kind of issue: [bug | feature-request-or-enhancement | Process Change | Support request]: Final fine tuning to the merge candidate: 233e8823201e8ea197e04c3caa9fabe6f773b907
Additional context
Last commit looks like our cold start performance is at parity with GPT2's pre-trained performance. It did run for 2 1/2 hours. Goals here:
Suggested Labels (If you don't know, that's ok): kind/performance kind/hpc hind/scientific