search

LSSTDESC / gcr-catalogs

A Python module that provides a unified interface to access mock galaxy catalogs and more for the LSST DESC
https://github.com/LSSTDESC/gcr-catalogs
BSD 3-Clause "New" or "Revised" License
26 stars 20 forks source link

Need to host DC1 catalog at NERSC and provide access via GCR? #314

Closed yymao closed 5 years ago

yymao commented 5 years ago

The dc1 catalog connects to the fatboy server at UW. But I'm getting this error:

DB-Lib error message 20009, severity 9:
Unable to connect: Adaptive Server is unavailable or does not exist (fatboy.phys.washington.edu:1433)
Net-Lib error during Connection timed out (110)

@danielsf do you know if this is an one-time issue or if the server no longer works? If the latter, should we update the connection info?

danielsf commented 5 years ago

heh....

Back in December, fatboy was observed mining bitcoin. UW IT decommissioned it permanently. I didn't say anything because I didn't think there were any DESC users of fatboy left.

We have reconstituted the database on a Linux workhorse in the UW Astronomy department using Microsoft SQL Server for Linux. We do not yet have a firm policy for off-campus access to this database, since the hosting machine is under heavy use by UW astronomers and we don't want to get in their way.

Do we think this DC1 catalog is something that people are going to actively use any more? If so, we might want to look at the possibility of hosting a copy of the DC1 galaxy catalog on NERSC, rather than depending on a remote connection to Seattle. There are only 17.5 million galaxies in that table (it is then dynamically tiled across the sky, but we will have python code to do that in lsst_sims shortly).

katrinheitmann commented 5 years ago

Pinging @fjaviersanchez in particular ...

yymao commented 5 years ago

Thanks @danielsf for the explanation and @katrinheitmann for pinging Javi. So this question is now whether we need to continue to provide access to DC1 galaxy catalog (i.e., host on NERSC, access via GCR).

But this also means that current dc1 catalog in GCRCatalog will not be supported anyway. I'll open a PR to remove it from GCRCatalog. And we can add back the new catalog on NERSC if people need it.

fjaviersanchez commented 5 years ago

Just to extend what I already commented in the #desc-dc2-users channel. I don't use GCR to access any of the DC1 data products (sorry). There are several DC1-related catalogs in the SSim project area including:

If there's enough interest, I can add these catalogs to GCR (or if there's someone interested in adding them to get the hang on how to use GCR that'd be great!).

yymao commented 5 years ago

Closing this since (1) there is no clear need of GCR access to DC1 catalog and (2) there's a copy of DC1 catalog on NERSC, so if a future need arises, we can write a new reader for it.

Thanks all.