search

camall3n / onager

Lightweight python library for launching experiments and tuning hyperparameters, either locally or on a cluster
MIT License
20 stars 4 forks source link

Current working directory agnostic onager #51

Open taodav opened 1 year ago

taodav commented 1 year ago

Currently, onager saves its prelaunch and history files and data in what is assumed to be the current working directory's .onager folder.

Ideally what should happen is that the .onager always references an .onager file in the project root (or the parent of the currently activated virtualenv).

This way you can run any onager command from any directory, as long as your virtualenv is activated.

camall3n commented 1 year ago

I like this idea. It would allow launching jobs from subdirectories, and it elegantly sidesteps the problem searching up and down the directory tree for where the .onager folder is.

However, I also see at least one potential issue...

Not everyone puts their virtual env in the code directory; some people use a single virtual env for multiple projects and put it somewhere like ~/venvs. This would mean job logs would be harder to access. Currently the workflow is

cd .onager/logs/slurm

If we made this change, the workflow for those people would get a lot more annoying:

cd ~/path/to/.onager/logs/slurm

This could be mitigated by something like an onager log command. We already have onager history and onager list, so we could potentially use similar functionality for something that allows viewing log files. But it might still slow things down since it wouldn't be able to do tab-completion.