search

niaid / image_portal_workflows

Workflows related to project previously referred to as "Hedwig"
BSD 3-Clause "New" or "Revised" License
5 stars 1 forks source link

Taking configs out of the source code #276

Closed annshress closed 1 year ago

annshress commented 1 year ago

Currently all the configs are hard-coded into the files for local development. https://github.com/niaid/image_portal_workflows/blob/main/em_workflows/config.py#L52C11-L52C11

Also, configs are edited at the server and left unstaged during deployment (? I think).

These configs should rely on some config file that should remain untracked and should exist in each computers separately. Example, developers with local config files, and remote server with remote config files and both having their independent configs.

For example: 

>>> dev.config (in local computer)

bioformats2raw='/opt/bin/bioformats2raw'
...

>>> server.config

bioformats2raw = "/gs1/apps/user/spack-0.16.0/spack/opt/spack/linux-centos7-sandybridge/gcc-8.3.1/bioformats2raw-0.7.0-7kt7dff7f7fxmdjdk57u6xjuzmsxqodn/bin/bioformats2raw"

Example: https://wiki.python.org/moin/ConfigParserExamples

mbopfNIH commented 1 year ago

I tried removing the machine-dependent paths by adding the Config.command_loc() function as part of Issue #158. It is still used in the local tests, but the shutil.which() command didn't work on the HPC. A config file is another way to handle this, but does need to be tracked in Git.

annshress commented 1 year ago

Thanks for the info Mike.

Refer to https://github.com/niaid/image_portal_workflows/issues/158.

mbopfNIH commented 1 year ago

This issue isn’t necessarily subsumed by the #158. That one was closed somewhat unsatisfactorily because of HPC setup issues that I couldn’t address at the time. An external config file could solve some of the problems that #158 couldn’t.

I defer to Philip on that decision as it would involve more runtime setup.