exalate-issue-sync[bot]
commented
1 year ago Arno Candel commented: Yes, let the user specify bounds on parameters, and let the black box sample from it
Example:
INPUT: Nsamples = 25
hidden=new int[]{100,100} // this is fixed in this example train_samples_per_iteration = 100 //fixed as well
// these are to be varied l1 in [0,1e-4] l2 in [0,1e-5] activation in {Tanh,TanhDropout,Rectifier} epochs in [1,100]
OUTPUT: 25 models with l1/l2/activation/epochs sampled (uniformly) from the above space, remaining parameters either fixed by user or default.
DinukaH2O
My plans: For Phase 1 I'm doing random with (essentially unlimited) explicit values, which can approximate true random with distributions. Stopping criteria will be: max_models, max_time_ms, and "metric of my choice hasn't improved more than 0.05 in the last 5 models". For Phase 2 I will add learning optimization with Gaussian Process Models. For Phase 3 I will add explicit hyperparameter lists (an ask from Mark). For Phase 4 I will add distributions for random and GP.