askap-vast / vast-tools

A collection of tools that are useful for the VAST project and for exploration of results from the VAST Pipeline.
https://vast-survey.org/vast-tools/
MIT License
8 stars 0 forks source link

Add check for existence of requested data to Query init #346

Closed ddobie closed 2 years ago

ddobie commented 2 years ago

Fix #339.

This code will check that the requested data exists at the initialisation of a Query object. It checks that the relevant directories exist and assumes that if those directories exist then the catalogues/fits files it should contain also exist.

The goal is a quick first-pass check to see if the data is available (since, e.g., nimbus doesn't have any Stokes Q/U/V data) with a nice exit if it doesn't. Adding a check that all files exist is quite cumbersome and probably unnecessary, as the query will still fail somewhat nicely (i.e. it includes the filepath in the traceback) if they don't.

This solution doesn't directly fix #339, mostly because the jupyterhub instances don't have unique hostnames and therefore a simple hostname check is not feasible. Restricting access to Stokes Q/U/V data to ada is not ideal as some users may have downloaded full stokes data to their own machines. However, this solution is more general and produces a similar outcome while also handling any other instances of missing data (e.g. there is currently no COMBINED data for some epochs on nimbus, there are a number of missing epochs on ada)