search

wmo-im / BUFR4

BUFR edition 4
MIT License
27 stars 9 forks source link

Table D sequence for reporting trajectory profile data from marine animal tags #17

Closed david-i-berry closed 3 years ago

david-i-berry commented 4 years ago

Branch

https://github.com/wmo-im/BUFR4/tree/issue-17-ft

Authors

Summary and purpose

The US Integrated Ocean Observing System (IOOS) Animal Telemetry Network (ATN) aims to start distributing temperature, depth and salinity measurements from marine animal tags over the GTS. A draft Table D sequence to meet the requirements to do so is presented for review and comment.

Action proposed

The team is kindly asked to review and provide comment on the contents of this proposal for future consideration of validation status approval.

Discussions

The US Integrated Ocean Observing System (IOOS) Animal Telemetry Network (ATN) Data Assembly Center (DAC) receives and curates real time and delayed mode data from tags attached to marine animals. These tags typically provide measurements of the sea surface temperature along a trajectory whist the animal is on the surface and periodic profiles from dives as the animals forage for food. The profiles include measurements of the temperature, pressure, and salinity.

A small subset of this data is distributed by the community over the GTS, but currently in the TAC TESAC format (FM-64). The IOOS ATN aims to start distributing more of this data over the GTS in real time and the draft template seeks to meet the requirements for doing so. Coordination within the community is ongoing.

Detailed proposal

New Table D Entries

Category 15: Oceanographic report sequences

Table Reference Table References Element Name Description
3-15-013 Sequence for reporting trajectory profile data from marine animal tags
3-01-150 WIGOS Identifier
0-01-087 WMO marine observing platform extended identifier WMO number where assigned
2-08-032 Change width of CCITT IA5 change width to 32 characters
0-01-019 Ship or mobile land station identifier Platform ID, e.g. ct145-933-BAT2-18 (max 32 characters)
2-08-000 Change width of CCITT IA5 Cancel change width
0-03-001 Surface station type 11 (Marine animal)
0-22-067 Instrument type of water temperature and / or salinity measurement set to 995 (attached to marine animal)
1-14-000 Delayed replication of 14 descriptors
0-31-001 Delayed descriptor replication factor
0-08-021 Time significance set to 26, time of last known position
3-01-011 (Year, month, day)
3-01-012 (Hour, minute)
3-01-021 (Latitude / longitude (high accuracy))
0-01-051 Platform transmitter ID number e.g. Argos PTT
0-02-148 Data collection and / or location system
0-01-012 Direction of motion of moving observing platform
0-01-014 Platform drift speed (high precision)
0-33-022 Quality of buoy satellite transmission
0-33-023 Quality buoy location
0-33-027 Location quality class (range of radius of 66% confidence)
0-07-063 Depth below sea / water surface
0-22-045 Sea / water temperature
0-08-021 Time significance Set to missing / cancel
1-07-000 Delayed replication of 7 descriptors
0-31-001 Delayed descriptor replication factor
3-01-011 (Year, month, day)
3-01-012 (Hour, minute)
3-01-021 (Latitude / longitude (high accuracy))
0-01-079 Unique identifier for the profile Profile ID
0-01-023 Observation sequence number Upcast number
0-22-056 Direction of profile Set to 0 (ascending / upwards)
3-06-035 Temperature and salinity profile

Additions to code tables

0-03-001
Surface station type

Code figureMeaning
0Land station (synoptic network)
1Shallow water station (fixed to sea/lake floor)
2Ship
3Rig/platform
4Moored buoy
5Drifting buoy (or drifter)
6Ice buoy
7Land station (local network)
8Land vehicle
9Autonomous marine vehicle
10Reserved
* 11Tag attached to marine animal
12 - 30Reserved for future use
31Missing value

New entry (11) indicated by *.

Attachments

Excel document showing expanded sequences atn_bufr_template_draft.xlsx

david-i-berry commented 4 years ago

Hi Enrico,

Sorry, I meant to flag this sooner. After the call I've been in conversation with the people behind the template and we'd like to aim for the next FT, not FT-2021-1. Would that be possible.

Thanks,

Dave.

david-i-berry commented 4 years ago

Updated to reflect changes as part of validation, move of PTT ID field and transmitter ID / location method out of first replication block. Revised sequence below, changes flagged using *** in description column:

New Table D Entries

Category 15: Oceanographic report sequences

Table Reference Table References Element Name Description
3-15-013 Sequence for reporting trajectory profile data from marine animal tags
3-01-150 WIGOS Identifier
0-01-087 WMO marine observing platform extended identifier WMO number where assigned
2-08-032 Change width of CCITT IA5 change width to 32 characters
0-01-019 Ship or mobile land station identifier Platform ID, e.g. ct145-933-BAT2-18 (max 32 characters)
2-08-000 Change width of CCITT IA5 Cancel change width
0-03-001 Surface station type 11 (Marine animal)
0-22-067 Instrument type of water temperature and / or salinity measurement set to 995 (attached to marine animal)
0-01-051 Platform transmitter ID number e.g. Argos PTT *** modified (move)
0-02-148 Data collection and / or location system *** modified (move)
1-12-000 Delayed replication of 12 descriptors *** modified (change from 14 to 12 descriptors
0-31-001 Delayed descriptor replication factor
0-08-021 Time significance set to 26, time of last known position
3-01-011 (Year, month, day)
3-01-012 (Hour, minute)
3-01-021 (Latitude / longitude (high accuracy))
0-01-012 Direction of motion of moving observing platform
0-01-014 Platform drift speed (high precision)
0-33-022 Quality of buoy satellite transmission
0-33-023 Quality buoy location
0-33-027 Location quality class (range of radius of 66% confidence)
0-07-063 Depth below sea / water surface
0-22-045 Sea / water temperature
0-08-021 Time significance Set to missing / cancel
1-07-000 Delayed replication of 7 descriptors
0-31-001 Delayed descriptor replication factor
3-01-011 (Year, month, day)
3-01-012 (Hour, minute)
3-01-021 (Latitude / longitude (high accuracy))
0-01-079 Unique identifier for the profile Profile ID
0-01-023 Observation sequence number Upcast number
0-22-056 Direction of profile Set to 0 (ascending / upwards)
3-06-035 Temperature and salinity profile
david-i-berry commented 4 years ago

Sample file and decoded values attached within zip archive.

Sample file (example-profile.bufr) encoded by Axiom using custom encoder and tables. Output from two different decoders

Expected values in attached Excel spread sheet, decoded values in agreement.

  1. sample and decoded values Archive.zip
  2. expected values example-profile.xlsx
  3. table B BUFRCREX_TableB_en.txt
  4. table D BUFR_TableD_en.txt
  5. table D for eccodes sequence.txt
david-i-berry commented 4 years ago

I've believe the sample validated as it's been successfully decoded using two independent pieces of software but both have been run by me.

david-i-berry commented 4 years ago

@chenxiaoxia2019 Please use the BUFRCREX_CodeFlag_en_03.csv and BUFR_TableD_en_15.csv from the attached zip file when creating the branch. There are no new Table B entries.

Archive.zip

chenxiaoxia2019 commented 4 years ago

@DavidBerryNOC Hi, David, a new branch for this issue has been created. Could you please check it? Thanks.

david-i-berry commented 4 years ago

@chenxiaoxia2019 the branch has been checked (both visually and in decoding the test data) and is correct, many thanks.

amilan17 commented 3 years ago

Please check validation status @SibylleK

david-i-berry commented 3 years ago

This was accepted as validated back in June. It missed the inclusion in FT-2020-2 drafting and I failed to pick that up (it's possible the card hadn't been moved to the correct column in the project board). I had sent an email on this a couple of weeks ago that I've been meaning to follow up on (and whether we could use the sequence pre-operationally).

SibylleK commented 3 years ago

Just to confirm the validation, here is the output of the DWD BUFR software using the branch for issue-17. example-profile.bufr_DWDout.txt The values correspond to the expected values.

amilan17 commented 3 years ago

@DavidBerryNOC 

https://github.com/wmo-im/BUFR4/tree/issue-17-FT 

Can you verify that the modification in this issue are correct and, if so, move to 'validated' column?  

(branch issue-17 is really out of sync with Master, this was easier for me than trying to fix conflicts found while trying to rebase)

david-i-berry commented 3 years ago

Branch

https://github.com/wmo-im/BUFR4/tree/issue-17-ft

is correct (updated on the 22nd Jan). The other branch (https://github.com/wmo-im/BUFR4/tree/issue-17-FT, updated last November) is not correct and does not include the new sequence.

david-i-berry commented 3 years ago

I'm unable to move to the validated column, this may be a permissions issue.

amilan17 commented 3 years ago

@DavidBerryNOC  - Thanks. I will delete issue-17-FT. Do the codes in issue-17-ft branch need revalidation?

SibylleK commented 3 years ago

I got the same result using the entries of branch issue-17-ft for the decoding of Davids example from last June.

david-i-berry commented 3 years ago

Thanks Sibylle!

jitsukoh commented 3 years ago

@amilan17 @efucile Do you have the approximate date for table publication and the implementation date for FT2021-1? As Enrico suggested at the meeting last week, if the new tables are published earlier, probably there is no need for using the pre-operational status. It was unfortunate that the issue fell into the gap during the transition of our team and I feel that we need to make this new sequence usable as soon as possible.