Closed sbailey closed 1 year ago
From my experiments, we can at least hide the values of the "Don't Add"s by using pushenv
and pushing an empty value:
pushenv TYCHO_DIR ""
and then they should be restored as expected.
We could also leave these alone, but it is sort of advertising the existence of things we don't necessarily want people asking us about.
For the record: DESI_BASIS_TEMPLATES
and DESI_MASK
are now in place.
Note, the implied path of QN_MODEL_FILE
actually puts it in ets
not edr
because edr/target/catalogs
is a pure symlink to ets/target/catalogs
. This is OK for this one file, but, e.g. dr1/target/catalogs
may need to be an actual directory with subdirectories linking to ets
.
@weaverba137 I added the surveyops sv2 and sv3 directories with the following commands as the desi user; these are ready for you to add checksums (after re-granting write access) and then you can close this.
# cd and re-grant ability to write to this dir
cd $CFS/desi/public/edr/survey/ops/surveyops/tags/0.1/mtl
chmod u+w .
# svn export sv2 and sv3
svn export https://desi.lbl.gov/svn/data/surveyops/tags/0.1/mtl/sv2
svn export https://desi.lbl.gov/svn/data/surveyops/tags/0.1/mtl/sv3
# grant world read + execute
chmod -R a+rX sv2
chmod -R a+rX sv3
# remove write permission, even from self
chmod -R a-w .
For the record: we will leave the environment variables pointing to the proprietary locations, which might have some use for DESI collaborators switching to desitree/edr . We can revisit that choice if it turns out to be problematic.
For documentation purposes, "module swap desitree/edr" appears to be the standard module syntax, though "module switch desitree/edr" appears to do the same thing. e.g. see https://lmod.readthedocs.io/en/latest/010_user.html which documents "swap" but not "switch".
@weaverba137 recreated the checksum file; closing the ticket.
Also note: the resulting edr
module file is archived as etc/edr.module
.
This is related to the environment variables in desihub/desimodules#47, but I'm starting a new ticket to focus on EDR data instead of software environments, and I'm putting it in desitree as the closest repo we have for data directory organization topics.
ADD TO EDR
Add the following directories to EDR and update desitree/edr to set these to the EDR location:
ADD TO EDR
These further augment $DESI_SURVEYOPS=/global/cfs/cdirs/desi/public/edr/survey/ops/surveyops/tags/0.1/ These exist in the tag, but not yet in the export (we only want to release a subset of the tag).
DON'T ADD
For the record, some other directories that we should not add to EDR. If we can find an easy way for desitree/edr to unset these but have them re-appear when swapping back to desitree/0.6.0 that would be nice. If that is tricky, we could leave them as-is (which means they would continue to work for DESI collaborators which is sort of a feature rather than a bug).
Reformatting of other surveys' data releases
These are needed as inputs for running target selection but not needed for using target selection outputs. Re-releasing them may have licensing / acknowledgements that's we'd have to clarify first
Working directories for survey ops
(and in some cases don't even exist because env is redefined elsewhere for real usage)
Will add to DR1
We'll want it for DR1, but it needs some cleanup first. Caveat: we may want to post-facto add it to EDR just to support more recent software releases, but even then it is only used for low level CCD preprocessing.
Unused