search

LSSTDESC / desc-data-portal

LSST DESC Data Portal web app and the associated documentation and example notebooks.
https://data.lsstdesc.org
BSD 3-Clause "New" or "Revised" License
7 stars 2 forks source link

Add instructions for non-DESC NERSC user #74

Open wmwv opened 3 years ago

wmwv commented 3 years ago

Would it be appropriate to add instructions to the Data Portal that are for someone who has a NERSC account but is not a member of DESC. These are implicitly there because they could just set up GCRCatalogs and such themselves and know that they can skip the download-the-data part, but for the greatest simplicity, could we add explicit instructions for such a user.

As an example, the DESI Legacy Survey provides download instructions in its data release and data model, but also describes the files directly on disk at NERSC: https://www.legacysurvey.org/dr9/files/

heather999 commented 3 years ago

Need to think about this further.. such a non-DESC NERSC user who doesn't download the data themselves, would need to point GCRCatalogs at our /global/cfs/cdirs/lsst/gsharing/lsstdesc area by setting their ROOT_DIR appropriately. In that scenario, they would only have access to what we provide in the gsharing area.

wmwv commented 3 years ago

Thanks. Yeah, it's not quite the same as the DESI Legacy Survey release where they made all of the files in their processing area globally readable. And I don't know how many NERSC users interested in cosmology are not DESC users, whereas there are many cosmology people who are not DESI members who might be interested in the files at NERSC.

wmwv commented 3 years ago

a non-DESC NERSC user who doesn't download the data themselves, would need to point GCRCatalogs at our /global/cfs/cdirs/lsst/gsharing/lsstdesc area by setting their ROOT_DIR appropriately. In that scenario, they would only have access to what we provide in the gsharing area.

But optimistically I think those two sentences, perhaps slightly rewritten, might essentially serve as the instructions.

yymao commented 3 years ago

Is /global/cfs/cdirs/lsst/gsharing globally readable? If so, that's similar to DESI, right?

If that's the case, I think we just need to add, on this "Install and Configure GCRCatalogs" page, under "Configure: Setting up root_dir for GCRCatalogs", that if they are on NERSC, they can do:

python -m GCRCatalogs.user_config set root_dir /global/cfs/cdirs/lsst/gsharing
heather999 commented 3 years ago

Yes /global/cfs/cdirs/lsst/gsharing is globally readable. Sounds good, as long as they're happy using just what we put there under gsharing - they can avoid doing any downloads and setting up a separate area at NERSC. Do we need to differentiate the instructions for DESC users who may not understand that they won't have access to everything DESC makes available this way?