search

LSSTDESC / rail

Top level "umbrella" package for RAIL
MIT License
8 stars 4 forks source link

conditional install #11

Closed aimalz closed 1 year ago

aimalz commented 1 year ago

When installing from source, we discussed looking into options for installing each standalone RAIL ecosystem package sequentially such that the whole installation wouldn't fail if one of them breaks, given that the code already makes use of introspection to accommodate if only some creators/estimators are installed. (Also I think the setup.py is defunct now that we have pyproject.toml, yes?)

eacharles commented 1 year ago
  1. depends on what you mean. pip won't let you do what you call conditional installs, but we can a should write a script that does that, and as well as a script that does an install from source of all the rail packages, and those should live here.
  2. As far as setup.py is concerned, it isn't defunct, it is just that all the configuration now lives in pyproject.toml. if you look in the setup.py file you will see that it is just a couple of lines.
eacharles commented 1 year ago

Chatting with Alex today, what I would actually recommend is building up a compound rail command line tool that lets you do things like :

rail install --conda-name rail-py3.10 --conda-reqs conda_reqs.txt --python-version=3.10
rail source-install --install-dir .. --conda-name rail-py3.10 --conda-reqs conda_reqs.txt --python-version=3.10
rail run BPZ_Lite --input <some_file> --model <some_file> --j 12 --config <some_file> --output <some_file>
rail run BPZ_Lite --help  # prints the configuration parameters, input and outputs for BPZ_Like
rail run pipeline.yml --input <some_file> --outdir .   # runs a pipeline
rail list-stages 
rail list-packages
rail render-nb 
rail clean-nb

If you have a favorite way to build up command line tools, go for it. If not, have a look here:

https://github.com/lsst-dm/cm_tools/tree/main/src/lsst/cm/tools/cli

All of the parameters available to any one of the subcommands are defined in options.py, the actual subcommands are defined in commands.py, note that most of them just grab stuff from the kwargs (which are parsed out from the command line) and pass them along to other functions. Happy to sit with you and explain it if that is helpful.

eacharles commented 1 year ago

closed with #30