search

wmo-im / BUFR4

BUFR edition 4
MIT License
27 stars 9 forks source link

Table D sequence for reporting marine observations from unmanned surface vehicles #8

Closed efucile closed 4 years ago

efucile commented 4 years ago

Branch

https://github.com/wmo-im/BUFR4/tree/issue-8

Summary and purpose

New sequence for the reporting of met-ocean observations from unmanned surface vehicles.

Action proposed

The meeting is requested to review and approve the contents of this proposal for validation status.

Discussions

Observations from unmanned, or autonomous, surface vehicles are currently converted to BUFR and inserted on to the GTS using the sequence for drifting buoys (315009). However, this sequence is inadequate for the full range of observable parameters and sensors on the surface vehicles. Additionally, the use of the incorrect template leads the observations being mis-attributed to drifting buoys, resulting in the location and platform motion being incorrectly flagged by quality control / assurance systems. The sequence proposed in this document seeks to address these issues.

Dr David Berry, representative from Joint WMO-IOC Technical Commission for Oceanography and Marine Meteorology (JCOMM), proposed a Table D sequence to address these issues as in the Annex to this paragraph.

Dr Berry hoped the proposal would be validated by November 2019 and operational by FT2020-1. ECMWF, NOAA/NWS and DWD will assist the validation.

Detailed proposal

Add new entry to BUFR Table D

Category 15 – Oceanographic report sequences

Table Reference Table References Element Name Element Description

FXXYYY 315011 |   | Met-ocean observations from unmanned surface vehicles |     | 301150 | (WIGOS identifier) |     | 001036 | Agency in charge of operating the observing platform |     | 001085 | Observing platform manufacturer’s model |     | 001086 | Observing platform manufacturer’s serial number |     | 003001 | Surface station type |     | 208032 | Change width of CCITT IA5 field |     | 001079 | Unique ID for profile | UUID for report, 32 character hex string   | 208000 | Change width of CCITT IA5 field |     | 301011 | Year, month, day |     | 301012 | Hour, minute |     | 301021 | (Latitude/longitude (high accuracy)) |     | 001012 | Direction of motion of moving observing platform |     | 001013 | Speed of motion of moving platform |     | 103000 | Delayed replication of 3 descriptor |     | 031000 | Short delayed descriptor replication factor |     | 007031 | Height of barometer above mean sea level |     | 306038 | Sequence for representation of standard surface marine meteorological observations from moored buoys |     | 012161 | Skin temperature |     | 101000 | Delayed replication of 1 descriptors |     | 031000 | Short delayed descriptor replication factor |     | 306034 | (Surface current) |     | 101000 | Delayed replication of 1 descriptor |     | 031000 | Short delayed descriptor replication factor |     | 306039 | (Sequence for representation of basic wave measurements) |     | 101000 | Delayed replication of 1 descriptors |     | 031000 | Short delayed descriptor replication factor |     | 306033 | (Surface salinity) |     | 101000 | Delayed replication of 1 descriptor |     | 031000 | Short delayed descriptor replication factor |     | 306041 | (Depth and temperature profile (high accuracy/precision)) |     | 101000 | Delayed replication of 1 descriptors |     | 031000 | Short delayed descriptor replication factor |     | 306004 | (Depth, temperature, salinity) |     | 101000 | Delayed replication of 1 descriptor |     | 031000 | Short delayed descriptor replication factor |     | 306005 | Sub-surface current measurements |     | 105000 | Delayed replication of 5 descriptors |     | 031000 | Short delayed descriptor replication factor |     | 041001 | pCO2 |     | 008043 | Atmospheric chemical or physical constituent type |     | 015028 | Mole fraction of atmospheric constituent / pollutant in dry air |     | 008043 | Atmospheric chemical or physical constituent type |     | 013080 | Water pH |     | 104000 | Delayed replication of 4 descriptors |     | 031000 | Short delayed descriptor replication factor |     | 041005 | Turbidity |     | 041003 | Dissolved nitrates |     | 022188 | Dissolved oxygen |     | 041002 | Fluorescence |     | 101000 | Delayed replication of 1 descriptor |     | 031000 | Short delayed descriptor replication factor |     | 306040 | (Sequence for representation of detailed spectral wave measurements) |     | 104000 | Delayed replication of 4 descriptors |     | 031000 | Short delayed descriptor replication factor |     | 008021 | Time significance |     | 004025 | Time period or displacement |     | 014017 | Instantaneous long-wave radiation |     | 014018 | Instantaneous short-wave radiation |  

Expanded proposed sequence 315011
Met-ocean observations from unmanned surface vehicles

Table References Element Name
FXXYYY1 FXXYYY2

Identification 301150 |   | (WIGOS identifier)   | 001125 | WIGOS identifier series   | 001126 | WIGOS issuer of identifier   | 001127 | WIGOS issue number   | 001128 | WIGOS local identifier (character) 001036 |   | Agency in charge of operating the observing platform 001085 |   | Observing platform manufacturer's model 001086 |   | Observing platform manufacturer's serial number 003001 |   | Surface station type 208032 |   | Change width of CCITT IA5 field 001079 |   | Unique ID for report 208000 |   | Change width of CCITT IA5 field Date, time, location and motion 301011 |   | Year, month, day   | 004001 | Year   | 004002 | Month   | 004003 | Day 301012 |   | Hour, minute   | 004004 | Hour   | 004005 | Minute 301021 |   | (Latitude/longitude (high accuracy))   | 005001 | Latitude (high accuracy)   | 006001 | Longitude (high accuracy) 001012 |   | Direction of motion of moving observing platform 001013 |   | Speed of motion of moving platform Basic meteorological observations 103000 |   | Delayed replication of 3 descriptor 031000 |   | Short delayed descriptor replication factor 007031 |   | Height of barometer above mean sea level 306038 |   | Sequence for representation of standard surface marine meteorological observations from moored buoys   | 010004 | Pressure   | 010051 | Pressure reduced to mean sea level   | 007033 | Height of sensor above water surface   | 012101 | Temperature/air temperature   | 012103 | Dewpoint temperature   | 013003 | Relative humidity   | 007033 | Height of sensor above water surface   | 008021 | Time significance   | 004025 | Time period or displacement   | 011001 | Wind direction   | 011002 | Wind speed   | 008021 | Time significance   | 004025 | Time period or displacement   | 011041 | Maximum wind gust speed   | 004025 | Time period or displacement   | 007033 | Height of sensor above water surface   | 002005 | Precision of temperature observation   | 007063 | Depth below sea/water surface (cm)   | 022049 | Sea-surface temperature 012161 |   | Skin temperature Surface currents 101000 |   | Delayed replication of 1 descriptors 031000 |   | Short delayed descriptor replication factor 306034 |   | (Surface current)   | 002031 | Duration and time of current measurement   | 003010 | Method of sea/water current measurement   | 002040 | Method of removing velocity and motion of platform from current   | 022005 | Direction of sea-surface current   | 022032 | Speed of sea-surface current Surface waves 101000 |   | Delayed replication of 1 descriptor 031000 |   | Short delayed descriptor replication factor 306039 |   | (Sequence for representation of basic wave measurements)   | 022078 | Duration of wave record   | 022070 | Significant wave height   | 022073 | Maximum wave height   | 022074 | Average wave period   | 022071 | Spectral peak wave period   | 022076 | Direction from which dominant waves are coming   | 022077 | Directional spread of dominant wave Surface salinity 101000 |   | Delayed replication of 1 descriptors 031000 |   | Short delayed descriptor replication factor 306033 |   | (Surface salinity)   | 002033 | Method of salinity/depth measurement   | 007063 | Depth below sea/water surface (cm)   | 022064 | Salinity Temperature profile 101000 |   | Delayed replication of 1 descriptor 031000 |   | Short delayed descriptor replication factor 306041 |   | (Depth and temperature profile (high accuracy/precision))   | 002032 | Indicator for digitization   | 102000 | Delayed replication of 2 descriptors   | 031001 | Delayed descriptor replication factor   | 007062 | Depth below sea/water surface   | 022043 | Sea/water temperature Temperature and salinity profile 101000 |   | Delayed replication of 1 descriptors 031000 |   | Short delayed descriptor replication factor 306004 |   | (Depth, temperature, salinity)   | 002032 | Indicator for digitization   | 002033 | Method of salinity/depth measurement   | 103000 | Delayed replication of 3 descriptors   | 031001 | Delayed descriptor replication factor   | 007062 | Depth below sea/water surface   | 022043 | Sea/water temperature   | 022062 | Salinity Current profile 101000 |   | Delayed replication of 1 descriptor 031000 |   | Short delayed descriptor replication factor 306005 |   | Sub-surface current measurements   | 002031 | Duration and time of current measurement   | 103000 | Delayed replication of 3 descriptors   | 031001 | Delayed descriptor replication factor   | 007062 | Depth below sea/water surface   | 022004 | Direction of current   | 022031 | Speed of current pCO2 105000 |   | Delayed replication of 5 descriptors 031000 |   | Short delayed descriptor replication factor 041001 |   | pCO2 008043 |   | Atmospheric chemical or physical constituent type 015028 |   | Mole fraction of atmospheric constituent / pollutant in dry air 008043 |   | Atmospheric chemical or physical constituent type 013080 |   | Water pH Dissolved nitrates, oxygen and turbidity 104000 |   | Delayed replication of 4 descriptors 031000 |   | Short delayed descriptor replication factor 041005 |   | Turbidity 041003 |   | Dissolved nitrates 022188 |   | Dissolved oxygen 041002 |   | Fluorescence Spectral wave measurements 101000 |   | Delayed replication of 1 descriptor 031000 |   | Short delayed descriptor replication factor 306040 |   | (Sequence for representation of detailed spectral wave measurements)   | 022078 | Duration of wave record   | 022082 | Maximum non-directional spectral wave density   | 106000 | Delayed replication of 6 descriptors   | 031001 | Delayed descriptor replication factor   | 022080 | Waveband central frequency   | 022069 | Spectral wave density   | 022086 | Mean direction from which waves are coming   | 022087 | Principal direction from which waves are coming   | 022088 | First normalized polar coordinate from Fourier coefficients   | 022089 | Second normalized polar coordinate from Fourier coefficients Radiation 104000 |   | Delayed replication of 4 descriptors 031000 |   | Short delayed descriptor replication factor 008021 |   | Time significance 004025 |   | Time period or displacement 014017 |   | Instantaneous long-wave radiation 014018 |   | Instantaneous short-wave radiation

Reference document: https://wmoomm.sharepoint.com/:w:/s/wmocpdb/EbjXX95DCrJDn838EdH98a0BWvJkCaMNKG37ocLadRcIVw?e=v1AtKi

Reference meeting: https://community.wmo.int/activity-areas/wmo-codes/meetings/ipet-cm-iii

david-i-berry commented 4 years ago

Hi Enrico,

I think this is in the wrong project and is a duplicate of https://github.com/wmo-im/BUFR4/issues/6.

Cheers,

Dave.

efucile commented 4 years ago

@DavidBerryNOC absolutely correct. I need to get used to the multiple repos

efucile commented 4 years ago

@DavidBerryNOC it's in the correct place now!

david-i-berry commented 4 years ago

As part of the validation process the sequence has been modified, the revised proposal can be found below.

As part of this name has been changed from unmanned surface vehicles to autonomous surface vehicles.

PROPOSAL (Details of proposal) Add new entry to BUFR Table D Category 15 – Oceanographic report sequences

Table Reference Table References Element Name Description
F XX YYY F XX YYY
3 15 011 Met-ocean observations from autonomous surface vehicles
  3 01 150 (WIGOS identifier)
  0 01 087 WMO marine observing platform extended identifier
  0 01 036 Agency in charge of operating the observing platform
  0 01 085 Observing platform manufacturer’s model
  0 01 086 Observing platform manufacturer’s serial number
  0 03 001 Surface station type
  2 08 032 Change width of CCITT IA5 field
  0 01 079 Unique ID for profile UUID for report, 32 character hex string
  2 08 000 Change width of CCITT IA5 field
  3 01 011 Year, month, day
  3 01 012 Hour, minute
  3 01 021 (Latitude/longitude (high accuracy))
  0 01 012 Direction of motion of moving observing platform
  0 01 014 Platform drift speed (high precision)
  0 11 104 True heading of aircraft, ship or other mobile platform[DB2]
  1 03 000 Delayed replication of 3 descriptor
  0 31 000 Short delayed descriptor replication factor
  0 07 031 Height of barometer above mean sea level
  3 06 038 Sequence for representation of standard surface marine meteorological observations from moored buoys
  0 12 161 Skin temperature
  1 01 000 Delayed replication of 1 descriptors
  0 31 000 Short delayed descriptor replication factor
  3 06 034 (Surface current)
  1 01 000 Delayed replication of 1 descriptor
  0 31 000 Short delayed descriptor replication factor
  3 06 039 (Sequence for representation of basic wave measurements)
  1 01 000 Delayed replication of 1 descriptors
  0 31 000 Short delayed descriptor replication factor
  3 06 033 (Surface salinity)
  1 01 000 Delayed replication of 1 descriptor
  0 31 000 Short delayed descriptor replication factor
  3 06 041 (Depth and temperature profile (high accuracy/precision))
  1 01 000 Delayed replication of 1 descriptors
  0 31 000 Short delayed descriptor replication factor
  3 06 004 (Depth, temperature, salinity)
  1 01 000 Delayed replication of 1 descriptor
  0 31 000 Short delayed descriptor replication factor
  3 06 005 Sub-surface current measurements
  1 05 000 Delayed replication of 5 descriptors
  0 31 000 Short delayed descriptor replication factor
  0 41 001 pCO2
  0 08 043 Atmospheric chemical or physical constituent type
  0 15 028 Mole fraction of atmospheric constituent / pollutant in dry air
  0 08 043 Atmospheric chemical or physical constituent type
  0 13 080 Water pH
  1 04 000 Delayed replication of 4 descriptors
  0 31 000 Short delayed descriptor replication factor
  0 41 005 Turbidity
  0 41 003 Dissolved nitrates
  0 22 188 Dissolved oxygen
  0 41 002 Fluorescence
  1 01 000 Delayed replication of 1 descriptor
  0 31 000 Short delayed descriptor replication factor
  3 06 040 (Sequence for representation of detailed spectral wave measurements)
  1 04 000 Delayed replication of 4 descriptors
  0 31 000 Short delayed descriptor replication factor
  0 08 021 Time significance
  0 04 025 Time period or displacement
  0 14 017 Instantaneous long-wave radiation
  0 14 018 Instantaneous short-wave radiation
chenxiaoxia2019 commented 4 years ago

@DavidBerryNOC Hi, David, I have already created the branch for this issue, could you please check it and review the table with new entries? Thanks.

david-i-berry commented 4 years ago

The addition to Table D (based on the merged file) is correct and I am able to use it in my decoding software and example file.

I'll upload samples shortly.

david-i-berry commented 4 years ago

Sample files (NetCDF, BUFR and decoded text) plus encoding software can be found at

https://github.com/DavidBerryNOC/bufr_saildrone

marijanacrepulja commented 4 years ago

Successfully decoded both sample files provided. Table D from branch issue-8 used in decoding with ecCodes.

saildrone-gen_5-Arctic-NOAA-2019-sd1033-20190717T100000-20190717T115959-1_minutes-v1.1563366702571.bufr.txt

saildrone-gen_5-tpos_pmel_2019_003-sd1066-20190717T120000-20190717T135959-1_minutes-v1.1563373962076.bufr.txt

david-i-berry commented 4 years ago

Thanks @marijanacrepulja.

With exception of truncation of decimal places for location elements and blank padding of strings in decoded output the output is identical to that expected.

Comparison of expected vs ecCodes given in attached files.

saildrone_comparison.xlsx

I've added the validated label as I believe this is now complete.

jbathegit commented 4 years ago

@DavidBerryNOC I was also able to read your saildrone-gen_5-tpos_pmel_2019_003-sd1066-20190717T120000-20190717T135959-1_minutes-v1.1563373962076.bufr using the NCEP BUFRLIB software, and a spot-check of the first 5 subsets agreed with the decoded values shown in your sample output listing.

amilan17 commented 4 years ago

SUMMARY: Add entries to Table D Category 15.

amilan17 commented 4 years ago

Approved by FT 2020-2.