taxi.sh directly edited by INSTALL.py

[dchackett]

Currently, INSTALL.py edits taxi.sh directly to tell it about TAXI_INSTALL_DIR. Is there some more graceful way of telling the taxi components where they are installed, without:

1. having a script directly edit a file, or
2. relying on environment variables like PATH, etc (Which, as a design feature, we should not assume anything about (on the run nodes) (hence, setting PYTHONPATH in taxi.sh))

[etneil]

I suspect the right way to do this and some of the other localization options is through a config file:

https://martin-thoma.com/configuration-files-in-python/

I think this is small and orthogonal to the refactor I'm doing, so I won't attempt to figure this out now.

[dchackett]

This does seem to be the obvious solution/a better way to organize a lot of things than what we're doing presently. However, there is the persisting issue of the taxi components not necessarily knowing where to find the config file.

Probably best solution: taxis take some command line argument like --cfgfn that tells them which config file to use.

Beyond TaxiInstallDir, such a config file could also specify how to call mpirun, which provides a neat solution to #1.

[etneil]

The way this more or less works in the redesign is that "local_taxi.py" functions as a sort of machine-specific configuration file, containing things like paths to all needed binaries, and the form of the mpirun command. If we have path issues to resolve, they could be addressed in there.

This is also dealt with in part by the migration to setuptools; this does make use of PATH and PYTHONPATH, but at least on some machines this works painlessly even for worker-node jobs. (For example, my custom PATHS were migrated successfully on the CU Beowulf machine in tests.)

Closing this issue for now, we'll see if it pops up again in some other form as we roll out on more remote machines...