camall3n / onager

Lightweight python library for launching experiments and tuning hyperparameters, either locally or on a cluster
MIT License
20 stars 4 forks source link

`+exclude` functionality! #54

Open taodav opened 1 year ago

taodav commented 1 year ago

As mentioned in #53, this PR is for the +exclude functionality. +exclude removes a subset of the runs generated by +args, by removing all runs that include arguments with the cross product of all +exclude hyperparams. So for example:

prelaunch +command test +jobname test +arg --test1 1 2 3 +arg --test2 4 5 6 +exclude --test1 1 2 +exclude --test2 5

Should exclude all jobs with arguments

I've also written a test according to this example, but am unsure of how to run the test...

camall3n commented 1 year ago

This looks useful and the code is clean. Nice job!

I'm willing to merge this in if you will also update the README to document how it works. I think a short description plus a worked example would suffice. The example should use real-sounding variable names, so it's clear intuitively what the use case is.

camall3n commented 1 year ago

Regarding how to run tests, currently it's python tests/test_prelaunch.py.

But I'm noticing that the prelaunch tests broke when I added the history subcommand, and I didn't notice because we never set up any CI actions.

camall3n commented 1 year ago

@taodav I don't like messing with the constants in this PR. We should discuss where the default .onager directory goes in a separate PR.

taodav commented 1 year ago

My bad! Forgot this PR was open and implemented it for my own use.

camall3n commented 1 year ago

Np. But let's finish it and merge at some point! :)

camall3n commented 1 year ago

Please remove your .idea folder

taodav commented 1 year ago

Just caught it, removed!

camall3n commented 1 year ago

Awesome. Can you provide a worked example in the readme as well?

taodav commented 1 year ago

Done! Also added titles for the types of arguments and options.