search

aodn / content

Tracks AODN Portal content and configuration issues
0 stars 0 forks source link

ANFOG Real-time filters are incorrect #294

Closed kereid closed 6 years ago

kereid commented 7 years ago

On step 1 select platform Glider then add IMOS- Australian National Facility for Ocean Gliders (ANFOG) - Real-time glider deployments.

On step 2 click the status drop down.

What should happen:

See filters either in_progress or completed and each one selected displays something on the map

What does happen:

See a filter titled delayed_mode and when it is selected there is nothing displayed on the map.

Have spoken to @bpasquer who will investigate

bpasquer commented 6 years ago

the issue comes from 4 files of the deployment Leuuwin20161013. These files have the status "delayed_mode". All the files from this deployment except these 4 files have been deleted from the schema after reception of this deployment data in delayed mode (harvested in ANFOG_DM schema) , hence their status. One solution to clean up the schema would be to re-push the files in the pipeline and then delete them with po_s3_del. However surprisingly, these file don't have a single version on S3 while the other files obviously do :

bpasquer@10-aws-syd:~/TEMPO$ s3lsv -a -b imos-data -k IMOS/ANFOG/REALTIME/seaglider/Leeuwin20161013/IMOS_ANFOG_BCEOPSTUV_20161013T071112Z_SG154_FV00_timeseries_END-20161128T175704Z.nc

Prefix: IMOS/ANFOG/REALTIME/seaglider/Leeuwin20161013/IMOS_ANFOG_BCEOPSTUV_20161013T071112Z_SG154_FV00_timeseries_END-20161128T175704Z.nc

s3lsv -a -b imos-data -k IMOS/ANFOG/REALTIME/seaglider/Leeuwin20161013/IMOS_ANFOG_BCEOPSTUV_20161013T071112Z_SG154_FV00_timeseries_END-20161124T122122Z.nc

Prefix: IMOS/ANFOG/REALTIME/seaglider/Leeuwin20161013/IMOS_ANFOG_BCEOPSTUV_20161013T071112Z_SG154_FV00_timeseries_END-20161124T122122Z.nc

                             N/A  2016-11-28T00:06:31.000Z  D  IMOS/ANFOG/REALTIME/seaglider/Leeuwin20161013/IMOS_ANFOG_BCEOPSTUV_20161013T071112Z_SG154_FV00_timeseries_END-20161124T122122Z.nc
9r2XnD240icTnLT8Bom8NA46aCrqdb9O  2016-11-24T18:02:07.000Z  K  IMOS/ANFOG/REALTIME/seaglider/Leeuwin20161013/IMOS_ANFOG_BCEOPSTUV_20161013T071112Z_SG154_FV00_timeseries_END-20161124T122122Z.nc
bpasquer@10-aws-syd:~/TEMPO$

Now I guess the only option to remove them from the ANFOG_RT schema is to delete these entries manually? @lwgordonimos

bpasquer commented 6 years ago

Solve by https://github.com/aodn/harvesters/commit/9cdbc91fa4de33ecdc8f2497249f644c1d818889#diff-4ffbeb04030e27763e853e3fcc657832