ROBUSTNESS: `./lg3.conf` must not be edited while running pipeline

[HenrikBengtsson]

If the ./lg3.conf file is edited while running the pipeline, then the changes will affect any scripts that are launched after the edit. In other words, the settings in ./lg3.conf are not frozen when the run script is called.

Suggestion

A few alternatives exist:

1. Only import the ./lg3.conf file in the run script and pass settings as environment variables.
2. Have the run script create a unique temporary copy of ./lg3.conf and have succeeding scripts use that.

I think the first approach is the best. It's a bit more tedious because it requires us to identify all env vars needed to be passed to qsub etc. OTH, that's a good thing and we already do this (somewhere or everywhere?).

[ivan108]

Let's consider alternative scenario:

• ${LG3_HOME}/lg3.conf contains links to software and resources and never edited by the user

• ./lg3.conf contains only Project level variables, such as EMAIL, PROJECT, CONV, ILIST/INTERVAL, LG3_CHASTITY_FILTERING, RECAL_BAM_EXT, and optionally overwrites some sofware and/or resources links, e.g. to replace the old Pindel with the new one. This file is edited only once before starting the project.

Both lg3.conf files are sourced by launching script and by PBS script using source_lg3_conf function. All those variables are exported into child processes using EXPORT. So, no need to source lg3.conf files any more by downstream scripts, nor pass any of those variables as arguments.

• Now run-time variables, which should be never included in lg3.conf files: PATIENT and/or SAMPLES + NORMAL, START. Those variables should be handled the same way as it is now: put into environment of the original shell and pass as argument to qsub. After that, we can continue passing it as argument or EXPORT into child shells.

• Special variable LG3_HOME should be treated same as run-time variables.

  What do you think? Do you see any obvious problems with this scenario? Thanks!

[HenrikBengtsson]

Let's punt on this one for now. We can get back to it later, after the C4 migration.