IMOS data reports - out of date

ggalibert commented 3 years ago

From issues created by lbesnard: aodn/issues#917

The SQL scripts generating the IMOS data reports (see https://github.com/aodn/reporting/tree/master/report/SQL_reporting_queries) are out of date. They don't represent the various schema changes which have occurred over the last few years, as it happened with the AATAMS SATTAG dataset earlier this year, ~~or with the current DWM migration~~.
The embargo plot in R is pointing to dw_aatams_acoustic instead of aatams_acoustic_reporting as detailed in https://github.com/aodn/PO-Backlog/issues/1389. dw_aatams_acoustic hasn't been updated since 2017.
They also don't take into account newly available collections, sub facilities, such as surface waves and others.

For reference, the IMOS data reports are then published at the following url: https://imos.org.au/data/data/data-reports

mhidas commented 3 years ago

The old ABOS reporting views have now been updated to DWM (https://github.com/aodn/reporting/pull/16)

lbesnard commented 2 years ago

@leonardolaiolo FYI, the old schema aatams_sattag_nrt should be decommissioned. See discussion here https://github.com/aodn/content/issues/468 The thing is that this schema is still used by the data reports https://github.com/aodn/reporting/blob/master/report/SQL_reporting_queries/aatams_sattag.sql#L31 which is a bit of a nonsense since the collection was apparently removed from the portal as well as the pipeline

Could you discuss this with @ggalibert and modify the SQL queries accordingly and let know @jachope when it will be done ?

aodn / content