Jacob-Stevens-Haas / gen-experiments

Pysindy experiments. Composable and extensible
MIT License
1 stars 2 forks source link

Update experiment to version 0.3.0 of mitosis #13

Closed Jacob-Stevens-Haas closed 10 months ago

Jacob-Stevens-Haas commented 10 months ago

This change removes the CLI from gen-experiments and uses the new generic CLI from mitosis. See examples in the experiment-scripts branch:

nohup mitosis gen_experiments.gridsearch \
    --eval-param seed=19 \
    --group lorenz \
    --folder trials \
    --param metrics=all \
    --param other_params=lor-ross-cubic \
    --param grid_params=duration-absnoise \
    --param grid_vals=duration-absnoise \
    --param grid_decisions=plot2 \
    --param series_params=kalman-auto3 \
    --param +plot_prefs=test-absrel5 \
    --param +skinny_specs=duration-noise &> lorenz.log &

Once installed, mitosis has a decent tooltip from mitosis --help/mitosis -h.

Why

The old version of mitosis loaded the parameters (which could use any arbitrary python objects) and sent them to the jupyter notebook via pickle. For complex objects like gurobipy optimizers (in MIOSR), this would cause errors. It also required some fancy loading logic in this repo. Now, parameters are split into two types: eval (like seed=19) and lookup (like metrics=all). The latter are simply defined in a regular python dictionary (or in this case, a nested dictionary).

Jacob-Stevens-Haas commented 10 months ago

Also @yb6599 , because another group is using mitosis too, I'm going to be pretty regular about releasing new mitosis versions on pypi.

yb6599 commented 10 months ago

So I can run experiments once this PR is merged? I was also going through the changes that come with this PR. Specifically, __init__.py removes all the different nested dictionaries such as sims_params, diff_params, feat_params, etc. Where do we access these parameters from, if they are removed from __init__.py? Do they come from mitosis directly?

yb6599 commented 10 months ago

For --param +plot_prefs=test-absrel5 \ --param +skinny_specs=duration-noise What does the '+' before plot_prefs and skinny_specs represent?

Jacob-Stevens-Haas commented 10 months ago

Good questions. Mitosis 0.3.0 added the +.

nested dictionaries such as sims_params, diff_params, feat_params, etc

These are now in config.py. See: vars(config)

So I can run experiments once this PR is merged?

yessir. But I want to give you practice with PR review, so if you have a moment, give it a full looksie, read the commit messages, comment on code, etc.

yb6599 commented 10 months ago

Now, parameters are split into two types: eval (like seed=19) and lookup (like metrics=all).

Are there more eval parameters other than seed?

Jacob-Stevens-Haas commented 10 months ago

Looks good! Looks like similar changes must be done in pdes.py for it to work properly.

Yep, good catch

Are there more eval parameters other than seed?

No. Well, technically group is handled internally as an eval parameter at points, but it's passed as its own command line argument.