search

aodn / public-schema

Shared schema specifications for data exchange (WFS, CSV, etc...)
GNU General Public License v3.0
1 stars 0 forks source link

Water Sampling Database - CPR data filter fails #82

Open leonardolaiolo opened 6 months ago

leonardolaiolo commented 6 months ago

This has been raised by Jake (IMOS). Jake was looking for PCI data from 01-07-2023 to 23-01-2024, these data are available on the 'Derived Indices' that it is possible to choose on Step 3. Here relevant email:

Hi Leo, The ‘No Data’ indicator appears on both the Phytoplankton and Zooplankton CPR data collection on the portal for anything after May 2023. As I was looking for data delivery of deployment undertaken in the current financial year, I originally search any data after 1 July 2023 (see screenshot below) image001

After having a look at the database I noticed that PCI data were present in the selected time frame (01-07-2023 to 23-01-2024) but not any of the phyto or zoopl data. I think that we probably set up the time filter to one of the other phyto or zooplankton parameter that are 'Null' in the time frame you chosen. To fix this we probably need to make some changes in Geoserver to set the time filter on PCI data that seems to come through first.

clairedavies commented 6 months ago

Can confirm that PCI data will always be first, as we do it first and we can't enter phyto or zoop data without it.

mhidas commented 5 months ago

I think that we probably set up the time filter to one of the other phyto or zooplankton parameter that are 'Null' in the time frame you chosen. To fix this we probably need to make some changes in Geoserver to set the time filter on PCI data that seems to come through first.

Something like that, but the problem is a little more tricky & annoying... all the _data layers have the same SampleTime_UTC set up, and this column also appears in the corresponding _map layer, allowing the map shown in step 2 of the portal to show what data are available. However, because the query for e.g. cpr_phytoplankton_map includes a join with cpr_phyto_raw, it will only include samples where phyto abundance data have been recorded. So you'd need to update these queries to change the above behaviour...

Having said all that, I don't think we have time to work on this right now.