desihub / nightwatch

DESI Nightwatch: online data quality assurance
BSD 3-Clause "New" or "Revised" License
2 stars 2 forks source link

links to the fiberassign QA for the tiles being observed in a given night are broken #224

Open moustakas opened 3 years ago

moustakas commented 3 years ago

The links to the fiberassign QA files in nightwatch for the set of tiles being observed that night are dead / broken, e.g. https://data.desi.lbl.gov/desi/target/fiberassign/tiles/trunk/021/fiberassign-021103.png

This issue was posted to Slack and the following discussion ensured (lightly edited):

@araichoor  Anand Raichoor @moustakas: I just see this message; I guess that's 
because those tiles have not been committed to svn yet (note that I would have 
thought that this will happen for at least any tile not observed yet, as we commit 
them only after they are observed).

David Schlegel Right.  I guess file a ticket.  Nightwatch at KPNO could point to 
the files in /data/tiles/SVN_tiles.  For Nightwatch running at NERSC, I’m not sure 
what we do.

Stephen Bailey is /global/cfs/cdirs/desi/target/fiberassign/holding_pen = 
https://data.desi.lbl.gov/desi/target/fiberassign/holding_pen still current and synced 
to the holding pen at KPNO?  Nightwatch needs a URL not a local file, and it doesn't 
have a particularly good way to know if something is in the holding pen vs. committed 
to svn (especially since that could change after the the nightwatch webpage is created), 
but we could potentially provide both links.

Eddie Schlafly No.  It probably makes sense to have an appropriate hourly cron job 
rsync from KPNO to NERSC.

Stephen Bailey  @Eddie Schlafly Is KPNO the guaranteed reference copy and a one-way 
sync is sufficient, or do we need to support a bi-directional sync or otherwise plan for 
both to be updated independently (even if that was done in error)?  i.e. mechanically 
if you need to run N>>1 tiles and do so at NERSC, do they go via the NERSC holding 
pen first, or do they go first to the KPNO holding pen and then would be synced back to 
NERSC?

Eddie Schlafly  So far the NERSC side has happened in an ad hoc fashion, and not really 
through that directory.  I think eventually we want everything at KPNO, and in the interim 
we'll continue doing the NERSC side via special one-off directories.  So fine to do a one 
way KPNO->NERSC sync.

Stephen Bailey  Thanks for clarification.  If nothing else, having a NERSC copy of the 
holding pen provides us with a URL to view; the files at KPNO aren't accessible that way.

It would be really helpful for the observers to be able to have this information during the night.

dstndstn commented 2 years ago

duplicated in #238

sybenzvi commented 1 year ago

Looked into this yesterday and created PR #307 to address it. It's straightforward to have Nightwatch at KPNO use a different link than Nightwatch at NERSC; this can be done with some JavaScript. But, the fiberassign images in /data/tiles/SVN_tiles are not currently viewable from a web address at KPNO. They could be linked into an addressable Nightwatch page, but bokeh makes it a pain to do this, so it's not solved yet.