zhengxiawu
commented
3 years ago Actually, the random forest is the HPO for an HPO method (NAS algorithm). "why does the accuracy steadily increase with each epoch that I train it for":random search will select the best architectures in the sampled architectures, therefore, it will steadily increase with each epoch. For the random forest model, we found that https://github.com/automl/fanova are doing exactly the same thing.Therefore, we strongly recommend users to use this code. We do not have much time to clear our random forest code, it's a mess.
impulsecorp
Your program works great, but I have a question about how exactly the epochs part of it works. I understand that your GitHub code does not include the random forest training, so when I run it like:
python augment.py --name=RS_BPE1 --file=random_darts_architecture.txt --data_path=data/ --save_path=experiment/ --batch_size=128 --lr=0.03 --layers=6 --init_channels=8 --epochs=600 --cutout_length=0 --image_size=16
I am using the results of your HPO search, and not actually training the random forest model to get those results. My question is, for Random Search, if it is just randomly training N models from the 100 architectures in the random_darts_architecture.txt file, using your pre-set hyperparameters, why does the accuracy steadily increase with each epoch that I train it for? What I mean is, if I run it for 600 epochs, what exactly is it doing in each epoch that increases the accuracy score? It can't be fine-tuning the random forest model, because that is not included in the repo.