Closed weaverba137 closed 1 month ago
Example of why this is somewhat tricky: I just merged desihub/desispec#2245 which reasonably wanted to update the desispec/main module to use desi_spectro_dark/latest instead of a named version. However, that has the consequence that we would need to temporarily undo that when we make the next desispec tag, which seems like a recipe for mistakes, e.g. this happened for a related case of desispec/0.63.x tagging desi_spectro_calib/trunk into the module files. As a consequence
source desi_environment.sh 24.4
--> DESI_SPECTRO_CALIB=...desi_spectro_calib/0.6.0
because that was set in desimodules/24.4, overriding desispec/0.63.0 setting it to trunk
module swap desispec/0.63.1
--> DESI_SPECTRO_CALIB=...desi_spectro_calib/trunk
because desispec overrode it back to trunk, which is probably not what we intended
I'm leaning towards saying that desimodules should own most of these environment variables, even for ones like DESI_SPECTRO_CALIB and DESI_SPECTRO_DARK which are only used by desispec.
Thoughts from others?
@sbailey, I am interested in your suggestion, but by "these" do you mean environment variables that are very, very similar to DESI_SPECTRO_CALIB
(point to a directory containing a version of a data set), or all of the environment variables in conflict listed above, or all "extra" environment variables generally?
After discussion with @sbailey today:
desimodules
:
a. Pointers to versioned data sets: DESI_SPECTRO_CALIB
, DESI_SPECTRO_DARK
, QN_MODEL_FILE
, etc.
b. NERSC performance: OMP_NUM_THREADS
, etc.
c. DUST_DIR
could also be defined in desimodules; that module is not desiInstall
-able so moving it to desimodules
would remove the need to hand-maintain that module.DESI_SURVEYOPS
from desitree
; though this is not as urgent.*_DIR
variables not explicitly mentioned in desitarget
will be left alone.
Environment variable audit
Introduction
desimodules
loads many packages, some of which define extra environment variables.desimodules
itself defines several environment variables.Finding environment variables in
desiconda
/desimodules
Find "interesting"
setenv
commands in all modulefiles:The sections below are ordered by the module load commands in
desimodules/24.4
(Jura).Environment variables that "point to the package" are ignored, e.g.
setenv RR_TEMPLATE_DIR $PRODUCT_DIR
.Summary of Conflicts
For each conflict, the order of the list of conflicting packages is the order in which they are loaded by
desimodules/24.4
.DUST_DIR
: desitarget, dust, fastspecfitDESI_SURVEYOPS
: desitree, desimodulesQN_MODEL_FILE
: desitarget, desispecSQ_MODEL_FILE
: desitarget, desispecDESI_SPECTRO_CALIB
: desispec, desimodulesMPICH_GNI_FORK_MODE
: desispec, redrock, fastspecfitKMP_AFFINITY
: desispec, redrock, fastspecfitOMP_NUM_THREADS
: redrock, fastspecfitAlthough these variables are not in conflict, we may want to think about centralizing where MP variables are defined to prevent future conflicts:
HDF5_USE_FILE_LOCKING
(redrock)MKL_NUM_THREADS
(fastspecfit)KMP_INIT_AT_FORK
(desimodules)Package Detail
desiconda
desitree
desitarget
desispec
desispec/main
anddesispec/0.63.0
have different values ofQN_MODEL_FILE
,0.63.0
is shown here:desisim
fiberassign
Note this is commented out:
SKYBRICKS_DIR
is defined bydesitarget
.redrock
dust
Multiple packages re-define
DUST_DIR
, so I'm including it here, even though it's a "generic"setenv
:fastspecfit
desimodules
These would be set after all other modules have been loaded:
desispec
setsDESI_SPECTRO_DARK
so that comment can be removed from the module file.