Implement NAO - Githubissues

Multi-Objective-NAS / self-supervised-nas

Official implementation of the paper "Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning" (NeurIPSW 2020)

5 stars 0 forks source link

Implement NAO #4

Closed juice500ml closed 3 years ago

juice500ml commented 3 years ago

[x] Implement bare minimum (Pre-existing baseline)
[x] Add tensorflow logging
[x] Implement loading pretrained weights
[x] Compare performance with the original implementation
[x] Add LR scheduler
[x] Implement cfg_validator
[ ] Find better hyperparameters
[x] Bug: generated_archs may contain duplicate seqs
[ ] Implement empty lr scheduler case
[ ] Implement accuracy

juice500ml commented 3 years ago

Random max (sampling 100): 0.9402 Orange: Semisupervised (max: 0.9485), Blue: Vanilla (max: 0.9403)

Random max (sampling 128): 0.9401 Our implementation (max: 0.9393, with different hyperparameters, suspecting inner_epochs is the culprit)

juice500ml commented 3 years ago

Nometric: Trained without metric loss Triplet: Trained with triplet loss

Freeze: Only train predictor Nofreeze: Train everything

Triplet loss clearly shows dominance!!!