Open MathewBiddle opened 6 months ago
The original task as listed above, i.e., provide open access table of all platforms with data submitted to the GTS has been completed. Table is available here: https://osmc.noaa.gov/erddap/tabledap/anibos_metadata.html. ATN platforms can be quired by the program field, select 'AniBOS ATN' or the country field, select 'United States'. These metadata are also being integrated into the OceanOPS dashboard. I update these metadata after new CTD deployments are registered in the ADR and BUFR flags embedded to trigger automated pathway the generates and serves BUFR messages to the NDBC stagging platform.
Great stuff. I can work with integrating that ERDDAP query into the ioos_metrics package. See https://github.com/ioos/ioos_metrics/issues/58
It would be good to gather some details on how data are getting to that ERDDAP. Is there any latency? Is it automated? What are the requirements on ATN's side to release data to the OSMC ERDDAP?
When looking at the ERDDAP response I see a lot less data than we are posting on https://stage-ndbc-bufr.srv.axds.co/platforms/atn/smru/profiles/. Any ideas about why there is a difference?
My understanding is that OMSC ERRDAP harvests data directly from the GTS. No requirement on ATN to release data to OMSC. But I can confirm this with Kevin O'Brien.
I'm not sure why only the 2024 metadata is appearing in the tableDAP service there definitely should be more deployments listed. I need to send Kevin some updated metadata from recent summer deployments, so I'll check on that as well.
AniBOS platform metadata has been updated to include all AniBOS ATN and AniBOS IMOS platforms with data submitted to the GTS from Dec 2019 through present.
Kevin O' Brien confirmed that OMSC harvests animal borne profile data submitted to the GTS as BUFR or TESAC messages automatically in near real-time, including ATN submissions and these data are available via OMSC ERDAPP.
On @MathewBiddle to add function to ioos_metrics.
Need to close these to close this:
Who is requesting this?
@ioos/marine-life
What is being requested?
Provide an open access table of all platforms with data submitted to the GTS including
WMO id
,deployment id
,ptt id
,platform code
anddeployment start date
.What is the requested deadline and why?
2025-09-30
What is the current status quo (i.e., what happens if this does not get done)?
The IOOS Office will not have readily available metrics about how many messages the ATN is sending to the GTS. These metrics are used to convey the importance of the system and identify impacts.
What indicates this is done (i.e., how do we know this is complete)?
Provide a description or any other important information.
We have some existing harvesting GTS statistics for ATN by scraping https://stage-ndbc-bufr.srv.axds.co/platforms/atn/smru/profiles/. This assumes that everything posted there makes it to the GTS. We would like to build in some monitoring capability to ensure what we think is going, is actually making it. Maybe we can harvest information from the OSMC ERDDAP https://osmc.noaa.gov/erddap/tabledap/OSMC_flattened.html
Below is a script for collecting the assumed metrics:
Could we check against the OSMC ERDDAP for platform = “TAGGED ANIMAL”:
https://osmc.noaa.gov/erddap/tabledap/OSMC_flattened.htmlTable?platform_id%2Cplatform_code%2Cplatform_type%2Ccountry%2Ctime%2Clatitude%2Clongitude%2Cobservation_depth%2Csst%2Catmp%2Cprecip%2Csss%2Cztmp%2Czsal%2Cslp%2Cwindspd%2Cwinddir%2Cwvht%2Cwaterlevel%2Cclouds%2Cdewpoint%2Cuo%2Cvo%2Cwo%2Crainfall_rate%2Chur%2Csea_water_elec_conductivity%2Csea_water_pressure%2Crlds%2Crsds%2Cwaterlevel_met_res%2Cwaterlevel_wrt_lcd%2Cwater_col_ht%2Cwind_to_direction%2Clon360&platform_type="TAGGED ANIMAL"