search

desihub / nightwatch

DESI Nightwatch: online data quality assurance
BSD 3-Clause "New" or "Revised" License
2 stars 2 forks source link

idea: load timeseries into telemetry DB #340

Open sbailey opened 1 year ago

sbailey commented 1 year ago

Passing on an idea from Klaus into a ticket: past efforts to implement timeseries plots for nightwatch (or desiperfapp) have gotten bogged down in the need to create and load a database so that the timeseries can be queried fast enough to usefully plot. But we already have a telemetry database containing a bunch of other timeseries and a graphana server at telemetry.desi.lbl.gov for plotting them. So the idea: create a script that loads nightwatch output into the existing opsdb so that we can leverage the existing graphana telemetry server for visualization? This could run as an optional last step for each event processed at KPNO (the NERSC copy would not attempt the loading, since the DB there is read-only). Or it could be a completely independent desi-7 cron job that runs every ~hour and loads any new exposures. @jose-bermejo @sybenzvi if you like this idea, coordinate with Klaus for access to the opsdb loading (and make sure credentials don't end up in the nightwatch repo etc. :)

sybenzvi commented 1 year ago

Not a bad idea... it would save quite a bit of time and effort. We'll talk to Klaus about it.