New Sequence and descriptors for GNSS Slant Total Delay BUFR

[SibylleK]

Branch

NEW: https://github.com/wmo-im/BUFR4/tree/issue18-FT

Summary and purpose

For the exchange of a large number of GNSS Slant Total Delay messages, a new STD BUFR template is proposed. The operational assimilation of STD observations is expected soon.

Action proposed

The team is invited to discuss and approve the proposed new descriptors and sequence for GNSS Slant Total Delay for inclusion within the next update to the WMO Manual on Codes. The team is also invited to participate in the validation process.

Discussions

Observations from global navigation satellite systems (GNSS), such as GPS, GLONASS, Galileo or BeiDou, provide valuable information about the atmospheric state and especially the distribution of water vapour. GNSS derived observations are currently used by most weather services which assimilate zenith total delays (ZTDs) operationally in global as well as regional numerical weather models. ZTDs are spatially and temporally averaged quantities which don‘t provide any information about vertical profiles. The successor of ZTDs are slant total delays (STDs) which provide information integrated along each individual satellite receiver path without the need of spatial averaging. The assimilation of many STDs observed from different directions and elevations improves the distribution of humidity fields in numerical weather models and leads to better forecasts. Currently, about 100 GNSS satellites are available and about 30 are always in view of each ground-based GNSS receiver. Large networks of GNSS receivers (see, e.g. egvap.dmi.dk) provide a large number of observations which scan the atmosphere from almost all directions.

A new STD BUFR template is required to store a large number of STDs with sufficient precision in an efficient way. While there is only one ZTD per station and epoch, up to 40 STDs can be observed, each STD observation consisting of more quantities than a single ZTD. The proposed STD BUFR template is able to store STDs as well as ZTDs and considers the needs of operational as well as scientific applications. As the assimilation of STDs is a rather new field of research it is important to provide additional information for validation and for improving the processing of STDs. The new format is based on the geodetic SINEX TRO Version 2.0 ASCII format (http://twg.igs.org/documents/sinex_tro_v2.00.pdf) used by the GNSS processing centres to provide the STD observations and can store all information relevant for atmospheric research without loss.

Several weather services are currently running STD assimilation experiments. The results are encouraging and the operational assimilation of STD observations is expected soon. Therefore, a commonly accepted format for exchanging STD observations is required.

Detailed proposal

Add new descriptors to BUFR Table B

FXY	ElementName	Note	Unit	Scale	RefVal	DataWidth	CREX_Unit	CREX_Scale	CREX_Width
015038	Path delay due to neutral atmosphere	(see Note 10)	m	4	0	20	m	4	11
015039	Estimated error in neutral atmosphere path delay	m	4	0	14	m	4	9
015079	Zenith path delay due to neutral atmosphere	(see Note 11)	m	4	0	15	m	4	9
015080	Estimated error in neutral atmosphere zenith path delay	m	4	0	12	m	4	8
015081	Wet path delay due to neutral atmosphere	(see Note 12)	m	4	0	18	m	4	10
015082	Path integrated water vapour	(see Note 16)	kg m-2	1	0	16	kg m-2	1	10
015083	GNSS derived neutral atmosphere gradient	(see Note 17)	m	5	0	14	m	5	9
015084	GNSS least squares residual	(see Note 18)	m	4	0	14	m	4	9
015085	GNSS multi-path delay	(see Note 15)	m	4	0	14	m	4	9
015086	GNSS hydrostatic mapping function	(see Note 19)	Numeric	3	0	16	Numeric	3	10
015087	GNSS wet mapping function	(see Note 19)	Numeric	3	0	16	Numeric	3	10
015088	GNSS gradient mapping function	(see Note 19)	Numeric	3	0	16	Numeric	3	10
015089	Zenith path delay due to neutral hydrostatic atmosphere	(see Note 13)	m	4	0	15	m	4	9
015090	Path delay due to neutral hydrostatic atmosphere	(see Note 14)	m	4	0	20	m	4	11
033093	Extended quality flags for ground-based GNSS data	Flag table	0	0	31	Flag table	0	31

Notes: (10) 015038 is the delay of an electromagnetic wave in the neutral atmosphere as compared to undisturbed propagation in vacuum. The delay due to ionized gases in the ionosphere is not covered, hence „neutral atmosphere“. The delay in meters is the time delay multiplied by the vacuum speed of light. (11) 015079 is the hypothetical path delay with the transmitter, e.g. GNSS satellite, in the zenith of the receiver. (12) 015081 is the contribution of atmospheric water vapour to the path delay. (13) 015089 is the contribution of the (almost) dry atmosphere to the zenith path delay as expressed by the hydrostatic equation. (14) 015090 is the zenith path delay due to the neutral hydrostatic atmosphere mapped onto the signal path. (15) 015085 is the excess delay due to multi-path effects which needs to be removed from the estimated path delay. (16) 015082 is the amount of atmospheric water vapour integrated along the signal path. (17) 015083 is the gradient (East-West or North-South) of the neutral atmosphere estimated by the GNSS processing. (18) 015084 Non-modelled contribution to the path delay estimated by the least squares adjustment of GNSS path delays. (19) A mapping function is an empirical projection of the zenith delay onto the path delay: Path delay = mapping function * zenith delay. It is a dimensionless real number greater than or equal to one. Different mapping functions are used for the wet and hydrostatic contributions to the delay and for the GNSS gradients.

Add a new Table D sequence 307024 - "Ground-based GNSS data - Slant Total Delay"

FXY	Elementname	Elementdescription
301150	WIGOS identifier
001015	Station or site name
001040	Processing centre ID code	GNSS processing centre
008021	Time significance	= 30 Time of occurrence
301011	"Year, month, day"
301013	"Hour, minute, second"
301022	"Latitude/longitude (high accuracy), height of station"
010036	Geoid undulation	"Geoid height above WGS-84 ellipsoid, ±150 m to 1 cm"
025061	Software identification and version number	"GNSS processing software, softwareversion"
010004	Pressure
012001	Temperature/air temperature
013003	Relative humidity
### Zenith Total Delay
120000	Delayed replication of 20 descriptors
031000	Short delayed descriptor replication factor
025060	Software identification
008021	Time significance	= 2 Time averaged
004025	Time period or displacement
115000	Delayed replication of 15 descriptors
031001	Delayed descriptor replication factor
015079	Zenith path delay due to neutral atmosphere
015080	Estimated error in neutral atmosphere zenith path delay
008022	Total number (with respect to accumulation or average)
033093	Extended quality flags for ground-based GNSS data
015089	Zenith path delay due to neutral hydrostatic atmosphere
015035	Component of zenith path delay due to water vapour
102002	Replicate 2 descriptors 2 times
008060	Sample scanning mode significance	= 5 North/South or =6 East/West
015083	GNSS derived neutral atmosphere gradient
201131	Change data width
202129	Change scale
013016	Precipitable water
202000	Change scale	Cancel
201000	Change data width	Cancel
015011	Log10 of integrated electron density
### Slant Total Delay
131000	Delayed replication of 31 descriptors
031000	Short delayed descriptor replication factor
025060	Software identification
008021	Time significance	Set to missing
004025	Time period or displacement	= 0
033093	Extended quality flags for ground-based GNSS data
125000	Delayed Replication of 25 descriptors
031001	Delayed descriptor replication factor	Number of GNSS satellites used
002020	Satellite classification
001050	Platform transmitter ID number
001150	Coordinate reference system	= 5 ECF coordinate system
202127	Change scale
304030	Location of platform
202000	Change scale
005021	Bearing or azimuth
007021	Elevation
015038	Path delay due to neutral atmosphere
015039	Estimated error in neutral atmosphere path delay
015090	Path delay due to neutral hydrostatic atmosphere
015081	Wet path delay due to neutral atmosphere
015082	Path integrated water vapour
015079	Zenith path delay due to neutral atmosphere
015089	Zenith path delay due to neutral hydrostatic atmosphere
015035	Component of zenith path delay due to water vapour
102002	Replicate 2 descriptors 2 times
008060	Sample scanning mode significance	= 5 North/South or = 6 East/West
015083	GNSS derived neutral atmosphere gradient
015084	GNSS least squares residual
015085	GNSS multi-path delay
015086	GNSS hydrostatic mapping function
015087	GNSS wet mapping function
015088	GNSS gradient mapping function
015011	Log10 of integrated electron density

*Add new Flag-table 033093 - Extended quality flags for ground-based GNSS data**

CodeFigure	EntryName
1	Path delay quality is considered poor
2	GALILEO satellites used
3	GLONASS satellites used
4	GPS satellites used
5	BeiDou satellites used
6-8	Reserved for new GNSS
9	Meteorological data applied
10	Atmospheric loading correction applied
11	Ocean tide loading applied
12	Second order ionosphere corrections applied
13	Third order ionosphere corrections applied
14	PPP solution
15	Gradients applied to path delay
16	Multipath corrections applied to path delay
17	Residual applied to path delay
18	Climate quality data processing
19	Re-processing
20	Post-processing
21	Near-real time data processing
22	Real time data processing
23-30	Reserved
All 31	Missing value

• GNSS = Global Navigation Satellite Systems

Add a new entry in Code Table 0 01 150 - Coordinate reference system

CodeFigure	EntryName
5	"Earth-Centred, Earth-Fixed (ECEF) coordinate system or Earth-Centred Rotational (ECR) system. This is a right-handed Cartesian coordinate system (X, Y, Z) rotating with the Earth. The origin is defined by the centre of mass of Earth. (Footnote (5) of class 27 does not apply if ECEF coordinates are specified.)"

[efucile]

Can we reformulate the proposal without notes?

[efucile]

Try to move the notes in a description column.

[SibylleK]

Due to @jbathegit proposal, I added the short delayed replication to the "Zenit total delay" and "Slant total delay block. My colleague Michael Bender will check whether the replication within the ZTD block is still needed.

[SibylleK]

The bit width of 033093 - "Extended quality flags for ground-based GNSS data" is changed from 32 to 31 (which is sufficient and does not create any problem when producing an example message for validation.)

The replication within the ZTD block is kept, as it does no harm and could be very useful for research and experimental purposes.

GNSS-STD.zip contains a BUFR message for the validation of 307024 and new table B and code table entries.

[jbathegit]

Just to document from today's TT-TDCF meeting, I am happy to help validate the samples once a test branch is available with the proposed new descriptors.

[amilan17]

@jbathegit @marijanacrepulja @sergioh-pessoal https://github.com/wmo-im/BUFR4/tree/issue-18

[SibylleK]

Dear @amilan17, Thank you very much for the branch. The proposed new entries are included in the branch. The Table B descriptors have to be changed (+6) as the new E-Profile descriptors in the upcoming version 35 are defined already as 015073 - 015078. I overlooked that.
The original proposal is changed accordingly.

I also included the notes of the descriptors in the Table B csv files as Note_en. But I am not 100% sure if this is acceptable, as they could become very long.

The output of the DWD BUFR software with the revised entries STD-Validation-BUFR-2.bufr.DWDoutRev.txt provide the same values as the output of the provided BUFR example. (GNSS-STD.zip)

@jbathegit @marijanacrepulja @sergioh-pessoal the text branch is now available with the proposed new descriptors. Thank you very much for any assistance in validating the GNSS STD BUFR proposal.

[jbathegit]

Using the values in @SibylleK 's updated branch, I was able to read the sample STD-Validation-BUFR-2.bufr file and decode identical values vs. what was shown in the sample STD-Validation-BUFR-2.bufr.out. So I think this is a successful validation using NCEP's BUFRLIB package.

STD-Validation-BUFR-2.bufr.debufr.txt

[sergioh-pessoal]

The decoding of the STD-Validation-BUFR-2.bufr file was also successful using the INPE software. The results were identical to the file STD-Validation-BUFR-2.bufr.out. STD-Validation-BUFR-2.bufr.INPE.txt

[marijanacrepulja]

I also successfully decoded the sample data using tables from the branch. STD-Validation-BUFR-2.bufr.txt

[amilan17]

@SibylleK -- I ran into merge conflicts when trying to rebase this branch. So I created a new branch and modified the relevant CSVs. Can you check it? 

https://github.com/wmo-im/BUFR4/tree/issue18-FT

[SibylleK]

Dear @amilan17, I have checked the new branch. It's OK, I didn't find any differences to the old branch. The output of the Validation-BUFR is exactly the same. Therefore the Validation of the new descriptors should be still valid.