Set up gen3 repo on NERSC CFS for general DESC access

LSSTDESC / ComputingInfrastructure

Gathering place for CI - Computing and Infrastructure - issues

3 stars 1 forks source link

Set up gen3 repo on NERSC CFS for general DESC access #59

Closed heather999 closed 3 years ago

heather999 commented 3 years ago

Jim requested that the Run2.2i and Run3.1i gen3 repos currently on /global/cscratch1/sd/descdm/Gen3/Run*i could be set up on CFS at NERSC.

The area will be set up under /global/cfs/cdirs/lsst/production/gen3/DC2/Run*i

To allow DESC users to easily utilize these repositories, they should be able to add symlinks to their own area (such as on CSCRATCH) under /global/cfs/cdirs/lsst/production/gen3/DC2/Run2.2i/run22_repo/u for example.

A globus copy is currently ongoing to copy the content of /global/cscratch1/sd/descdm/Gen3/Run*i to CFS. When complete, permissions need to be set up to allow all members of our DESC lsst Unix group the ability to add these symlinks.

heather999 commented 3 years ago

The globus copy completed. On Slack, Jim indicated that Run3.1i/run31_repo could be removed and we have no interest in the contents of Run2.2i/run22_repo/u So those have been removed.

The /global/cfs/cdirs/lsst/production/gen3 area allows full access to any desc* collaboration account, including descdm, so that should allow others to clean up as necessary.

I've set up /global/cfs/cdirs/lsst/production/gen3/DC2/Run2.2i/run22_repo/u to allow write by anyone in the lsst Unix group (all DESC NERSC users) and I've tested that will allow creation of symlinks in the u directory while preventing full access to the rest of the directory structure. Once run31_repo is set up , we'll want to do the same thing and set the permissions appropriately.

jchiang87 commented 3 years ago

I did some housekeeping on the new repositories to make sure they are usable for Gen3 processing:

Run2.2i:

Cleaned up unneeded files that were copied from the descdm scratch area.
Copied missing symlinks to the ref_cat/cal_ref_cat folder.
Made symlinks from descdm and jchiang8 scratch areas in run22i_repo/u folder.

Run3.1i:

Cleaned up unneeded files that were copied from descdm scratch area.
Built new run31i_repo using lsst_distrib w_2021_39.
Ingested Run3.1i WFD visits.
Set permissions for the run31i_repo/u folder to allow users in the lsst group to symlink their own output areas.

I also tested both repos by running DRP processing on the data in each.

jchiang87 commented 3 years ago

I think we're good to go here. Closing.