Closed Jacob-Stevens-Haas closed 10 months ago
Ok it's good to go at this point. Changes to the calling signature are just the inclusion of an indexing tuple to locate a gridsearch result. Existing experimental code should still work, e.g.
nohup python -m gen_experiments gridsearch --seed 19 \
--param ex_name=sho \
--param metrics=all \
--param other_params=4nonzero-cubic \
--param grid_params=duration-absnoise \
--param grid_vals=duration-absnoise \
--param grid_decisions=plot2 \
--param series_params=kalman-auto3 \
--param plot_prefs=test-absrel5 \
--param skinny_specs=duration-noise &> sho.log &
Sorry Yash, this is a twofer. I added it as a draft PR, so no review needed needed until I mark it ready for review. Just read and understand this. Ask questions about anything, like types, that you could use clarity on.
plot_prefs
argument ingridsearch.run
lets you choose which parameters in the gridsearch get their own plots. This means that we can't directly compare the individual plots that come out, because Kalman includes its own parameter identification methods, but other methods require specifying parameters BEFORE gridsearching to find the optimal parameters. This PR moves plotting the individual results until AFTER gridsearching and allows specifying the "optimal" parameter in aPlotPrefs
.