LSSTDESC / gcr-catalogs

A Python module that provides a unified interface to access mock galaxy catalogs and more for the LSST DESC
https://github.com/LSSTDESC/gcr-catalogs
BSD 3-Clause "New" or "Revised" License
26 stars 20 forks source link

Automatic detection of site fails for Perlmutter #610

Closed JoanneBogart closed 1 year ago

JoanneBogart commented 1 year ago

Existing code tries to determine if it's running at NERSC by looking for a particular pattern in the node name. This worked for cori but Perlmutter node names are undistinguished (e.g. login34) and don't satisfy the pattern used. One can establish site manually by means of the environment variable DESC_GCR_SITE. gcr-catalogs will use this value but still puts out a warning since the bogus "site" determined by the code doesn't match the value in the environment variable. Since our standard kernels at NERSC set DESC_GCR_SITE appropriately, most users don't need to be concerned with it beyond knowing to ignore the warning.
I propose we eliminate the code which tries to deduce site or, at the very least, eliminate the warning message.

heather999 commented 1 year ago

I haven't looked at the code that does this checking but one could just look for the existence of the env variable $NERSC_HOST to see if they are at NERSC or not.

JoanneBogart commented 1 year ago

Where $NERSC_HOST come from? Can we be confident it will always get set for any process at NERSC (in a batch job, in a container,..)? If so, that would be a big improvement over what we have now: it would generally do the right thing for NERSC users. However, it still won't work properly for any other supported site - currently in2p3 and nersc_public (as has been true all along). nersc_public is a special case which is probably only of interest to a few people who do testing for the data portal so doesn't concern me. At in2p3 I assume they would continue to get the warning message. New proposal: We could substitute the check for $NERSC_HOST for the code that's there now and also eliminate the warning.