wmo-im / BUFR4

BUFR edition 4
MIT License
27 stars 9 forks source link

Proposal for new BUFR table entries for EPS-SG : MWI, ICI, SCA, IAS 1, MWS #136

Closed antoinemerle closed 1 year ago

antoinemerle commented 2 years ago

Initial request

This document proposes new BUFR entries for the following products : MWI, ICI, SCA, IAS 1, MWS. This entries correspond to the instruments embeded for EPS-SG mission

Amendment details

New table D entries

New element Element name
3 10 080 Ice Cloud Imager Data (level 1B)
3 10 081 Microwave Imager Data (level 1B)
3 10 082 Microwave Sounder Data (level 1B)
3 10 083 Metop-SG Scatterometer (SCA), Sigma zero full resolution level 1B (SZF)
3 10 084 Metop-SG Scatterometer (SCA), Sigma zero resampled level 1B (SZR)
3 10 085 IASI-NG L1C (Radiances)
3 10 086 IASI-NG L1D (Principle Component Scores)

ICI

Add the following sequence to BUFR TABLE D : Add the following sequence `3 10 080 "Ice Cloud Imager Data (level 1B)" : Table references Element name note
0 01 033 Identification of originating/generating centre  
0 01 034 Identification of originating/generating sub-centre  
0 01 007 Satellite identifier  
0 02 019 Satellite instruments  
0 05 040 Orbit number  
0 25 061 Software identification and version number  
2 01 133 Add 8 to descriptor width  
0 05 041 Scan line number  
2 01 000 Cancel change of width  
3 01 011 Year, month, day  
3 01 012 Hour, minute  
2 07 003 Increase scale, reference value and data width (+3)  
0 04 006 Second  
2 07 000 Cancel increase scale, reference value and data width  
2 02 126 Add -2 to descriptor scale  
0 07 001 Height of station Spacecraft altitude
2 02 000 Cancel change of scale  
0 05 063 Spacecraft roll  
0 05 064 Spacecraft pitch  
0 05 066 Spacecraft yaw  
0 05 077 Sub-satellite point latitude  
0 06 043 Sub-satellite point longitude  
0 06 041 Earth scan sample number  
0 06 042 Space/moon scan sample number  
0 08 098 Source of temperature measurement Onboard calibration target (0)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Space view reflector (1)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Main reflector (2)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Sun shield (4)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Cancel significance (15)
0 07 074 Solar elevation angle in instrument reference frame  
0 05 076 Solar azimuth in instrument reference frame  
2 01 124 Add -4 to descriptor width  
2 02 127 Add -1 to descriptor scale  
0 25 084 Geometric rotational angle  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 33 100 ICI temperatures flag  
0 33 102 ICI scan quality flag  
0 33 104 ICI navigation status flag  
0 33 105 ICI processing flags  
1 13 007 Repeat 13 descriptors 7 times  
0 05 078 Horn number  
0 05 001 Latitude (high accuracy)  
0 06 001 Longitude (high accuracy)  
0 07 024 Satellite zenith angle  
0 05 021 Bearing or azimuth  
0 07 025 Solar zenith angle  
0 05 022 Solar azimuth  
0 07 073 Angle between moon and space view  
2 01 132 Add 4 to descriptor width  
2 02 129 Add 1 to descriptor scale  
0 21 166 Land fraction  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
1 17 003 Repeat 17 descriptors 3 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 101 ICI calibration flag  
0 33 103 ICI data quality flag  
0 02 104 Antenna polarization  
2 01 131 Add 3 to descriptor width  
2 02 136 Add 8 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 002 Repeat 17 descriptors 2 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 101 ICI calibration flag  
0 33 103 ICI data quality flag  
0 02 104 Antenna polarization  
2 01 132 Add 4 to descriptor width  
2 02 136 Add 8 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 003 Repeat 17 descriptors 3 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 101 ICI calibration flag  
0 33 103 ICI data quality flag  
0 02 104 Antenna polarization  
2 01 133 Add 5 to descriptor width  
2 02 136 Add 8 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 003 Repeat 17 descriptors 3 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 101 ICI calibration flag  
0 33 103 ICI data quality flag  
0 02 104 Antenna polarization  
2 01 133 Add 5 to descriptor width  
2 02 136 Add 8 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 002 Repeat 17 descriptors 2 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 101 ICI calibration flag  
0 33 103 ICI data quality flag  
0 02 104 Antenna polarization  
2 01 131 Add 3 to descriptor width  
2 02 135 Add 7 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  

MWI

Add the following sequence `3 10 081" Microwave Imager Data (level 1B)" : Table references Element name note
0 01 033 Identification of originating/generating centre  
0 01 034 Identification of originating/generating sub-centre  
0 01 007 Satellite identifier  
0 02 019 Satellite instruments  
0 05 040 Orbit number  
0 25 061 Software identification and version number  
2 01 133 Add 8 to descriptor width  
0 05 041 Scan line number  
2 01 000 Cancel change of width  
3 01 011 Year, month, day  
3 01 012 Hour, minute  
2 07 003 Increase scale, reference value and data width (+3)  
0 04 006 Second  
2 07 000 Cancel increase scale, reference value and data width  
2 02 126 Add -2 to descriptor scale  
0 07 001 Height of station Spacecraft altitude
2 02 000 Cancel change of scale  
0 05 063 Spacecraft roll  
0 05 064 Spacecraft pitch  
0 05 066 Spacecraft yaw  
0 05 077 Sub-satellite point latitude  
0 06 043 Sub-satellite point longitude  
0 06 041 Earth scan sample number  
0 06 042 Space/moon scan sample number  
0 08 098 Source of temperature measurement Onboard calibration target (0)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Space view reflector (1)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Main reflector (2)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Racetrack (3)
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement Cancel significance (15)
0 07 074 Solar elevation angle in instrument reference frame  
0 05 076 Solar azimuth in instrument reference frame  
2 01 124 Add -4 to descriptor width  
2 02 127 Add -1 to descriptor scale  
0 25 084 Geometric rotational angle  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 33 095 MWI temperatures flag  
0 33 097 MWI scan quality flag  
0 33 106 MWI processing flags  
0 33 099 MWI navigation status flag  
1 13 007 Repeat 13 descriptors 8 times  
0 05 078 Horn number  
0 05 001 Latitude (high accuracy)  
0 06 001 Longitude (high accuracy)  
0 07 024 Satellite zenith angle  
0 05 021 Bearing or azimuth  
0 07 025 Solar zenith angle  
0 05 022 Solar azimuth  
0 07 073 Angle between moon and space view  
2 01 132 Add 4 to descriptor width  
2 02 129 Add 1 to descriptor scale  
0 21 166 Land fraction  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
1 17 001 Repeat 17 descriptors 1 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 131 Add 3 to descriptor width  
2 02 138 Add 10 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 002 Repeat 17 descriptors 2 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 132 Add 4 to descriptor width  
2 02 138 Add 10 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 003 Repeat 17 descriptors 3 times 333  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 132 Add 4 to descriptor width  
2 02 138 Add 10 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 003 Repeat 17 descriptors 3 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 131 Add 3 to descriptor width  
2 02 137 Add 9 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 002 Repeat 17 descriptors 2 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 132 Add 4 to descriptor width  
2 02 137 Add 9 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 003 Repeat 17 descriptors 3 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 133 Add 5 to descriptor width  
2 02 137 Add 9 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 003 Repeat 17 descriptors 3 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 131 Add 3 to descriptor width  
2 02 136 Add 8 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  
1 17 002 Repeat 17 descriptors 2 times  
0 05 042 Channel number  
2 02 131 Add 3 to descriptor scale  
0 02 153 Satellite channel centre frequency  
0 02 154 Satellite channel band width  
2 02 000 Cancel change of scale  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
0 33 096 MWI calibration flag  
0 33 098 MWI data quality flag  
0 02 104 Antenna polarization  
2 01 131 Add 3 to descriptor width  
2 02 136 Add 8 to descriptor scale  
0 14 045 Channel radiance  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 12 158 Noise-equivalent delta temperature while viewing cold target  
0 12 159 Noise-equivalent delta temperature while viewing warm target  

MWS

Add the following sequence 3 10 082 " Microwave Sounder Data (level 1B)" to BUFR Table D: Table references Element name note
0 08 070 Vertical sounding product qualifier  
0 01 033 Identification of originating/generating centre  
0 01 034 Identification of originating/generating sub-centre  
0 01 007 Satellite identifier  
0 02 019 Satellite instruments  
0 05 040 Orbit number  
0 25 061 Software identification and version number  
2 01 133 Add 8 to descriptor width  
0 05 041 Scan line number  
2 01 000 Cancel change of width  
0 05 043 Field of view number  
0 06 042 Space/moon scan sample number  
3 01 011 Year, month, day  
3 01 012 Hour, minute  
2 07 003 Increase scale, reference value and data width (+3)  
0 04 006 Second  
2 07 000 Cancel increase scale, reference value and data width  
3 01 021 Latitude/longitude (high accuracy)  
1 07 002 Repeat 7 descriptors 2 times  
0 05 080 Radius resolution  
2 01 132 Add 4 to descriptor width  
2 02 129 Add 1 to descriptor scale  
0 21 166 Land fraction  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 07 007 Height Terrain height
2 02 126 Add -2 to descriptor scale  
0 07 001 Height of station Spacecraft altitude
2 02 000 Cancel change of scale  
0 07 024 Satellite zenith angle  
0 05 021 Bearing or azimuth  
0 07 025 Solar zenith angle  
0 05 022 Solar azimuth  
2 01 124 Add -4 to descriptor width  
2 02 127 Add -1 to descriptor scale  
0 25 084 Geometric rotational angle  
2 02 000 Cancel change of scale  
2 01 000 Cancel change of width  
0 07 073 Angle between moon and space view  
0 12 070 Warm load temperature  
0 12 064 Instrument temperature  
0 08 098 Source of temperature measurement  
0 12 101 Temperature/air temperature  
0 08 098 Source of temperature measurement  
0 33 107 MWS scan line processing flags  
0 33 108 MWS navigation status flags  
0 33 109 MWS overall quality flags  
1 18 024 Repeat 18 descriptors 24 times
0 05 042 Channel number
1 04 004 Repeat 4 descriptors 4 times
0 05 079 Band number
2 02 130 Change scale (from -8 to -6)
0 02 154 Satellite channel band width
2 02 000 Change scale (reset to -8)
0 02 104 Antenna polarization  
0 02 153 Satellite channel centre frequency  
0 25 077 Bandwidth correction coefficient 1  
0 25 078 Bandwidth correction coefficient 2  
2 07 002 Increase scale, reference value and data width (+2)  
0 12 063 Brightness temperature  
2 07 000 Cancel increase scale, reference value and data width  
2 07 001 Increase scale, reference value and data width (+1)  
0 33 089 Noise equivalent delta temperature (NEdT) quality indicators for warm target calibration  
0 33 090 NEdT quality indicators for cold target calibration  
2 07 000 Cancel increase scale, reference value and data width  
0 33 110 MWS channel processing flags  

SCA SZF

Add the following sequence 3 10 083 "Metop-SG Scatterometer (SCA), Sigma zero full resolution level 1B (SZF)" to BUFR Table D Table references Element name note
0 01 033 Identification of originating/generating centre  
0 01 034 Identification of originating/generating sub-centre  
0 25 060 Software identification  
0 01 007 Satellite identifier  
0 02 019 Satellite instruments  
1 27 012 Replicate 27 descriptors 12 times  
0 05 040 Orbit number  
2 01 130 Change data width by adding +2  
0 06 034 Cross-track cell number  
2 01 000 Reset data width  
0 05 034 Along-track row number  
3 01 021 Latitude/longitude (high accuracy)  
0 08 012 Land/sea qualifier  
3 01 011 Year, month, day  
3 01 012 Hour, minute  
0 04 007 Seconds within a minute (microsecond accuracy)  
0 08 085 Beam identifier  
0 03 030 Antenna status  
0 02 104 Antenna polarization  
0 03 030 Antenna status  
0 02 104 Antenna polarization  
0 03 029 Swath  
0 08 075 Ascending/descending orbit qualifier  
2 02 129 Change scale  
2 01 131 Change data width  
0 02 111 Radar incidence angle  
2 01 000 Change data width  
2 02 000 Change scale  
0 05 021 Bearing or azimuth  
0 21 062 Backscatter  
0 21 026 ASCAT sigma-0 usability  
0 21 027 Land contribution ratio  
0 33 111 SCA quality flags  

SCA SZR

For 3 10 084 : Metop-SG Scatterometer (SCA), Sigma zero resampled level 1B (SZR) : Table references Element name note
0 01 033 Identification of originating/generating centre  
0 01 034 Identification of originating/generating sub-centre  
0 25 060 Software identification  
0 01 007 Satellite identifier  
0 02 019 Satellite instruments  
0 05 040 Orbit number  
0 06 034 Cross-track cell number  
0 05 034 Along-track row number  
3 01 021 Latitude/longitude (high accuracy)  
3 01 011 Year, month, day  
3 01 012 Hour, minute  
0 04 007 Seconds within a minute (microsecond accuracy)  
0 14 058 Corrected cross-pole backscatter  
0 33 112 Faraday rotation angle (one way)  
1 17 005 Replicate 17 descriptors 5 times  
0 08 085 Beam identifier  
0 08 012 Land/sea qualifier  
0 03 030 Antenna status  
0 02 104 Antenna polarization  
0 03 030 Antenna status  
0 02 104 Antenna polarization  
0 08 075 Ascending/descending orbit qualifier  
2 02 129 Change scale  
2 01 131 Change data width  
0 02 111 Radar incidence angle  
2 01 000 Change data width  
2 02 000 Change scale  
0 05 021 Bearing or azimuth  
0 21 062 Backscatter  
0 21 026 ASCAT sigma-0 usability  
0 21 027 Land contribution ratio  
0 25 056 Normalised error estimate in backscatter measurement (Kp)  

IASI-NG L1C RAD

Add the following sequence 3 10 085 "IASI-NG L1C (Radiances)" to BUFR Table D : Table references Element name note
0 01 007 Satellite identifier  
3 01 005 Originating centre/sub-centre  
0 02 019 Satellite instruments  
0 02 020 Satellite classification  
0 04 001 Year  
0 04 002 Month  
0 04 003 Day  
0 04 004 Hour  
0 04 005 Minute  
2 02 131 Change scale  
2 01 138 Change data width  
0 04 006 Second  
2 01 000 Change data width  
2 02 000 Change scale  
0 05 001 Latitude (high accuracy)  
0 06 001 Longitude (high accuracy)  
0 07 024 Satellite zenith angle  
0 05 021 Bearing or azimuth  
0 07 025 Solar zenith angle  
0 05 022 Solar azimuth  
0 05 043 Field of view number  
0 05 040 Orbit number  
2 01 133 Change data width  
0 05 041 Scan line number  
2 01 000 Change data width  
2 02 126 Change scale  
0 07 001 Height of station Spacecraft altitude
2 02 000 Change scale  
0 33 113 IASI-NG general quality flags  
1 05 170 Repeat 5 descriptor 170 times  
1 04 100 Repeat 4 descriptor 100 times  
2 01 137 Change data width  
0 05 042 Channel number  
2 01 000 Change data width  
0 14 046 Scaled radiance  
0 02 019 Satellite instruments  
1 10 007 Repeat 10 descriptors 7 times  
0 05 060 Y angular position from centre of gravity  
0 05 061 Z angular position from centre of gravity  
0 20 083 Amount of segment covered by scene  
1 06 020 Repeat 6 descriptors 20 times  
0 05 042 Channel number  
0 08 023 First-order statistics  
0 14 044 Channel radiance  
0 08 023 First-order statistics  
0 14 044 Channel radiance  
0 08 023 First-order statistics  
0 08 072 Pixel type  
0 20 083 Amount of segment covered by scene  
0 08 029 Surface type  
0 20 083 Amount of segment covered by scene  

IASI-NG L1D PCS

Add the following sequence 3 10 086 "IASI-NG L1D (Principle Component Scores)" to BUFR Table D : Table references Element name note
0 01 007 Satellite identifier  
3 01 005 Originating centre/sub-centre  
0 02 019 Satellite instruments  
0 02 020 Satellite classification  
0 04 001 Year  
0 04 002 Month  
0 04 003 Day  
0 04 004 Hour  
0 04 005 Minute  
2 02 131 Change scale  
2 01 138 Change data width  
0 04 006 Second  
2 01 000 Change data width  
2 02 000 Change scale  
0 05 001 Latitude (high accuracy)  
0 06 001 Longitude (high accuracy)  
0 07 024 Satellite zenith angle  
0 05 021 Bearing or azimuth  
0 07 025 Solar zenith angle  
0 05 022 Solar azimuth  
0 05 043 Field of view number  
0 05 040 Orbit number  
2 01 133 Change data width  
0 05 041 Scan line number  
2 01 000 Change data width  
2 02 126 Change scale  
0 07 001 Height of station Spacecraft altitude
2 02 000 Change scale  
0 33 113 IASI-NG general quality flags  
1 09 004 Replicate 9 descriptors 4 times  
0 25 140 Start channel  
2 01 129 Change data width  
0 25 141 End channel  
2 01 000 Change data width  
0 40 026 Score quantization factor  
0 40 016 Residual RMS in band  
1 01 000 Delayed replication of 1 descriptor  
0 31 002 Extended delayed descriptor replication factor  
0 40 017 Non-normalized principal component score  
0 02 019 Satellite instruments  
1 10 007 Repeat 10 descriptors 7 times  
0 05 060 Y angular position from centre of gravity  
0 05 061 Z angular position from centre of gravity  
0 20 083 Amount of segment covered by scene  
1 06 020 Repeat 6 descriptors 20 times  
0 05 042 Channel number  
0 08 023 First-order statistics  
0 14 044 Channel radiance  
0 08 023 First-order statistics  
0 14 044 Channel radiance  
0 08 023 First-order statistics  
0 08 072 Pixel type  
0 20 083 Amount of segment covered by scene  
0 08 029 Surface type  
0 20 083 Amount of segment covered by scene  

New entries in Table B :

Add the following elements to BUFR Table B :

F-X-Y Element name Unit Scale Reference value Bit width Instrument
0-03-029 Swath Code Table 0 0 3 SCA SZF
0-03-030 Antenna status Code Table 0 0 3 SCA SZF, SCA SZR
0-05-076 Solar azimuth in instrument reference frame Degrees 2 0 16 ICI, MWI
0-05-077 Sub-satellite point latitude Degrees 4 -900000 21 ICI, MWI
0-05-078 Horn number Numeric 0 0 4 ICI, MWI
0-05-079 Band number Numeric 0 0 6 MWS
0-05-080 Resolution radius m -3 0 6 MWS
0-06-041 Earth scan sample number Numeric 0 0 11 ICI, MWI
0-06-042 Space/moon scan sample number Numeric 0 0 6 ICI, MWI, MWS
0-06-043 Sub-satellite point longitude Degrees 4 -1800000 22 ICI, MWI
0-07-073 Angle between moon and space view Degrees 2 0 15 ICI, MWI, MWS
0-07-074 Solar elevation angle in instrument reference frame Degrees 2 -9000 15 ICI, MWI
0-08-098 Source of temperature measurement Code Table 0 0 4 ICI, MWI, MWS
0-14-058 Corrected cross-pole backscatter dB 2 -5000 13 SCA SZR
0-21-026 ASCAT sigma-0 usability Code Table 0 0 3 SCA SZF, SCA SZR
0-21-027 Land contribution ratio Numeric 4 0 14 SCA SZF, SCA SZR
0-25-056 Normalised error estimate in backscatter measurement (Kp) Numeric 4 0 14 SCA SZR
0-33-095 MWI temperatures flag Flag Table 0 0 8 MWI
0-33-096 MWI calibration flag Flag Table 0 0 16 MWI
0-33-097 MWI scan quality flag Flag Table 0 0 16 MWI
0-33-098 MWI data quality flag Flag Table 0 0 9 MWI
0-33-099 MWI navigation status flag Flag Table 0 0 16 MWI
0-33-100 ICI temperatures flag Flag Table 0 0 8 ICI
0-33-101 ICI calibration flag Flag Table 0 0 16 ICI
0-33-102 ICI scan quality flag Flag Table 0 0 16 ICI
0-33-103 ICI data quality flag Flag Table 0 0 9 ICI
0-33-104 ICI navigation status flag Flag Table 0 0 16 ICI
0-33-105 ICI processing flags Flag Table 0 0 16 ICI
0-33-106 MWI processing flags Flag Table 0 0 16 MWI
0-33-107 MWS scan line processing flags Flag Table 0 0 16 MWS
0-33-108 MWS navigation status flags Flag Table 0 0 16 MWS
0-33-109 MWS overall quality flags Flag Table 0 0 16 MWS
0-33-110 MWS channel processing flags Flag Table 0 0 9 MWS
0-33-111 SCA quality flags Flag Table 0 0 32 SCA SZF
0-33-112 Faraday rotation angle (one way) Deg 2 -18000 16 SCA SZR
0-33-113 IASI-NG general quality flags Flag Table 0 0 16 IASI-NG RAD, PCS
0-33-114 Out of bounds check Flag Table 0 0 17 IASI-NG L2 TWV

New entries in existing code and flag table

  Code Table 0-02-020: Satellite classification
Action Value New meaning
Add 62 EUMETSAT Polar System – Second Generation (EPS-SG)
  Code Table 0-02-104: Antenna polarization
Action Value New meaning
Add 8 Cross pole average
Update 9-14 Reserved
  Code Table 0-08-075: Ascending/descending orbit qualifier
Action Value New meaning
Update 2 Mixture of ascending and descending

New flag tables and related code

Code Table 0-03-029: Swath
Value Meaning
0 Left hand side
1 Right hand side
2 to 6 Reserved
7 Missing
Code Table 0-03-030: Antenna status
Value Meaning
0 Transmitting
1 Receiving
2 to 6 Reserved
7 Missing
Code Table 0-08-098: Source of temperature measurement
Value Meaning
0 On-board calibration target
1 Space view reflector
2 Main reflector
3 Racetrack
4 Sun shield
5 Rotating reflector
6 to 14 Reserved
15 Missing
Code Table 0-21-026: ASCAT sigma-0 usability
Value Meaning
0 Nominal
1 Close to nominal
2 Far from nominal
3 to 6 Reserved
7 Missing
Flag Table 0-33-095 : MWI temperatures flag
Bit Meaning
1 PRTs or THMs temperatures are missing or with anomalous readings
2 PRT temperatures of OBCT used in the radiometric calibrations is missing or with anomalous readings
3 THMs temperatures of SVR used in the radiometric calibrations are missing or with anomalous readings
4 THMs temperatures of Main Reflector are missing or with anomalous readings
5 THMs temperatures of the Racetrack are missing or with anomalous readings
6 THMs temperatures of the Receivers are missing or with anomalous readings
7 Reserved
All 8 Missing
Flag Table 0-33-096: MWI calibration flag
Bit Meaning
1 Radiometric calibration failed or is degraded
2 OBCT view counts averaged value over scans is missing (this is valid for the high-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
3 Cold space view counts averaged value over scans is missing (this is valid for the low-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
4 OBCT view counts averaged value over scans is degraded due to missing or anomalous counts values (this is valid for the high-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
5 Cold space view counts averaged value over scans is degraded due to missing or anomalous counts values (this is valid for the low-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
6 OBCT view radiance averaged value over scans is missing (this is valid for the high-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
7 Cold space view radiance averaged value over scans is missing (this is valid for the low-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
8 OBCT view radiance averaged value over scans is degraded due to missing or anomalous values (this is valid for the high-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
9 Cold space view radiance averaged value over scans is degraded due to missing or anomalous values (this is valid for the low-reference calibration load when the back-up calibration is applied for channels MWI-1 to MWI-3)
10 Non-linearity parameter computed using noise diodes is missing (meaningful only for channels MWI-1 to MWI-3)
11 Non-linearity parameter computed using noise diodes is degraded (meaningful only for channels MWI-1 to MWI-3)
12 PRTs or THMs temperatures of current scan are missing or with anomalous readings
13 Moon intrusion in cold space degraded calibration (less than Nvalid_c valid samples are not affected by Moon for the considered channel and scan)
14 Back-up calibration with noise diodes performed (meaningful only for MWI-1 to MWI-3)
15 Degraded Receiver temperatures lead to degraded calibration
All 16 Missing
Flag Table 0-33-097 : MWI scan quality flag
Bit Meaning
1 scan is degraded in raw data record
2 Time sequence error
3 Current scan is acquired after a gap
4 Scan is in period of initialization of calibration of data averages over scans (because of cold start/cold end, missing warm/cold L0 Packets or data gaps).
5 Moon intrusion angle in space view below threshold for at least one channel
6 Moon correction is applied but is degraded for at least one channel
7 Sun glint angle below threshold for at least one channel
8 RFI contamination in the Earth view or RFI Packet missing for the considered scan (only for MWI-1V and MWI-1H)
9 Satellite manoeuvre occurs in current scan
10 Missing warm or cold L0 Packets for the considered scan
11-15 Reserved
All 16 Missing
Flag Table 0-33-098 : MWI data quality flag
Bit Meaning
1 MWI spectral radiance data of channel is missing or degraded
2 Earth view counts of channel within scan are missing or out of bounds
3 Radiometric calibration failed or is degraded
4 Geolocation of channel is erroneous or degraded
5 NEΔT of this data granule is above threshold
6 MR emissivity and spillover correction failed or degraded
7 MR sidelobe correction failed or degraded
8 Channel is defective
All 9 Missing
Flag Table 0-33-099 : MWI navigation status flag
Bit Meaning
1 geolocation of channel is erroneous or degraded
2 Time sequence error
3 Missing or corrupted NAVATT file lead to the use of predicted orbit files
4 NAVATT Attitude data is degraded
5 Time correlation error (Missing IERS Bulletin)
6 Invalid ephemeris or attitude data
7 Satellite manoeuvre occurs in current scan
8 Non-nominal attitude with yaw, pitch, roll error above threshold Delta_YPR (nominally these angles are 0.0 deg for YSM)
9 Sampling time not within prescribed limits
10 Scan velocity not within prescribed limits
11 Bad pointing (sensor LOS does not intersect ellipsoid or not compliant with observation azimuth/elevation limits)
12 Invalid solar azimuth/zenith angles computed
13 DEM geolocation not performed (with use_DEM_flag = 1)
14 Error in Land fraction computation
15 Predicted Orbit File not available
All 16 Missing
Flag Table 0-33-100: ICI temperatures flag
Bit Meaning
1 PRTs or THMs temperatures are missing or with anomalous readings
2 PRT temperatures of OBCT used in the radiometric calibrations is missing or with anomalous readings
3 PRT temperatures of SVR used in the radiometric calibrations are missing or with anomalous readings
4 PRT temperatures of IRP and Sun Shield are missing or with anomalous readings
5 THMs temperature of Back-end are missing or with anomalous readings
6 THMs temperature of Front-end are missing or with anomalous readings
7 PRT temperatures of Main Reflector are missing or with anomalous readings
All 8 Missing
Flag Table 0-33-101 : ICI calibration flag
Bit Meaning
1 Radiometric calibration failed or is degraded
2 OBCT view counts averaged value over scans is missing
3 Cold space view counts averaged value over scans is missing
4 OBCT view counts averaged value over scans is degraded due to missing or anomalous counts values
5 Cold space view counts averaged value over scans is degraded due to missing or anomalous counts values
6 OBCT view radiance averaged value over scans is missing
7 Cold space view radiance averaged value over scans is missing
8 OBCT view radiance averaged value over scans is degraded due to missing or anomalous values
9 Cold space view radiance averaged value over scans is degraded due to missing or anomalous values
10 PRTs or THMs temperatures of current scan are missing or with anomalous readings
11 Moon intrusion in cold space degraded calibration (less than Nvalid_c valid samples are not affected by Moon for the considered channel and scan)
12 Degraded or missing Back-end Temperatures lead to degraded calibration
13-15 Reserved
All 16 Missing
Flag Table 0-33-102 : ICI scan quality flag
Bit Meaning
1 scan is degraded in raw data records
2 Time sequence error
3 Current scan is acquired after a gap
4 Scan is in period of initialization of calibration of data averages over scans (because of cold start/cold end, missing warm/cold L0 Packets or data gaps).
5 Moon intrusion angle in space view below threshold for at least one channel
6 Moon correction is applied but is degraded for at least one channel
7 Sun glint angle below threshold for at least one channel
8 Satellite manoeuvre occurs in current scan
9 Missing warm or cold L0 Packets for the considered scan
10-15 Reserved
All 16 Missing
Flag Table 0-33-103: ICI data quality flag
Bit Meaning
1 ICI spectral radiance data of channel is missing or degraded
2 Earth view counts of channel within scan are missing or out of bounds
3 Radiometric calibration failed or is degraded
4 Geolocation of channel is erroneous or degraded
5 NEΔT of this data granule is above threshold
6 MR emissivity and spillover correction failed or degraded
7 MR sidelobe correction failed or degraded
8 Channel is defective
All 9 Missing
Flag Table 0-33-104 : ICI navigation status flag
Bit Meaning
1 geolocation of channels is erroneous or degraded
2 Time sequence error
3 Missing or corrupted NAVATT file lead to the use of predicted orbit files
4 NAVATT Attitude data is degraded
5 Time correlation error (Missing IERS Bulletin)
6 Invalid ephemeris or attitude data
7 Satellite manoeuvre occurs in current scan
8 Non-nominal attitude with yaw, pitch, roll error above threshold Delta_YPR (nominally these angles are 0.0 deg for YSM)
9 Sampling time not within prescribed limits
10 Scan velocity not within prescribed limits
11 Bad pointing (sensor LOS does not intersect ellipsoid or not compliant with observation azimuth/elevation limits)
12 Invalid solar azimuth/zenith angles computed
13 DEM geolocation not performed (with use_DEM_flag = 1)
14 Error in Land fraction computation
15 Predicted Orbit File not available
All 16 Missing
Flag Table 0-33-105 : ICI processing flags
Bit Meaning
1 Moon contamination correction on cold space view counts is not applied
2 MR spillover correction relative to emission by instrument platform is not applied
3 SVR spillover correction relative to emission by instrument platform is not applied
4 SVR sidelobe correction is not applied
5 Full cross-polarization correction including small angles correction is applied
6 Dynamic sidelobe correction is not applied for ICI-1
7 Dynamic sidelobe correction is not applied for ICI-2
8 Dynamic sidelobe correction is not applied for ICI-3
9 Dynamic sidelobe correction is not applied for ICI-4 V and ICI-4 H
10-15 Reserved
All 16 Missing
Flag Table 0-33-106 : MWI processing flags
Bit Meaning
1 Moon contamination correction on cold space view counts is not applied
2 Calibration with noise diodes is not applied for at least one channel among MWI-1 to MWI-3
3 MR spillover correction relative to emission by instrument platform is not applied
4 SVR spillover correction relative to emission by instrument platform is not applied
5 SVR sidelobe correction is not applied
6 Full cross-polarization correction including small angles correction is applied
7 RFI correction in the Earth view is not applied
8 Dynamic sidelobe correction is not applied for MWI-1
9 Dynamic sidelobe correction is not applied for MWI-2
10 Dynamic sidelobe correction is not applied for MWI-3
11 Dynamic sidelobe correction is not applied for MWI-4
12 Dynamic sidelobe correction is not applied for MWI-8
13-15 Reserved
All 16 Missing
Flag Table 0-33-107 : MWS scan line processing flag
Bit Meaning
1 Non-nominal scan due to gap
2 Non-nominal warm target temperature
3 Non-nominal along-track averaging of warm target temperatures
4 Non-nominal instrument temperature
5 Non-nominal along-track averaging of instrument temperatures
6 Non-nominal rotating reflector temperature
7 Non-nominal along-track averaging of rotating reflector temperatures
8 Non-nominal antenna scan angle Earth view
9 Non-nominal antenna scan angle cold view
10 Non-nominal antenna scan angle warm view
11 Non-nominal geolocation
12 Non-nominal LSM and DEM assignment
13 Ongoing satellite manoeuvre
14 Redundant 50GHz rec and PRT set in use
15 Reserved
All 16 Missing
Flag Table 0-33-108 : MWS navigation status flag
Bit Meaning
1 Non-nominal condition
2 Scan not processed
3 Geolocation not successful
4 Orbit not initialised with NAVATT
5 Attitude not initialised with NAVATT
6 Predicted Orbit File ingest error
7 Manoeuvre
8 IERS Bulletin A ingest error
9 Time correlation and EOP initialised with Predicted Orbit File
10 Full accuracy of NAVATT initialized attitude not reached
11 Pointing mode not YSM
12 NAVATT ingest error
13 to 15 Reserved
All 16 Missing
Flag Table 0-33-109 : MWS overall quality flags
Bit Meaning
1 Missing input product(s)
2 Data gap(s)
3 Corrupted input product(s)
4 Instrument anomaly
5 Missing or degraded auxiliary data
6 Degraded due to manoeuvre
7-15 Reserved
All 16 Missing
Flag Table 0-33-110 : MWS channel processing flags
Bit Meaning
1 Channel is off
2 Non-nominal mean warm view count
3 Non-nominal along-track averaging of warm view counts
4 Non-nominal mean cold view count
5 Non-nominal along-track averaging of cold view counts
6 Non-nominal calibration coefficients
7 Moon intrusion in some of the cold view FOVS
8 Non-nominal processing or excessive value for NEDT (cold or warm)
All 9 Missing
Flag Table 0-33-111 : SCA quality flags
Bit Meaning
1 Predicted orbit file used
2 Manoeuvre taking place
3 Reserved
4 Satellite is not in yaw steering mode
5 Transponder signal is present
6 Noise estimate is poor
7 Noise estimate is very poor
8 Noise outlier is present
9 Ancillary data is poor
10 Ancillary data is very poor
11 Characterisation table limits exceeded
12 Power-gain estimate is poor
13 Power-gain estimate is very poor
14 Power-gain out of limits
15 Waveguide loss estimate is poor
16 Waveguide loss estimate is very poor
17 Waveguide table limits exceeded
18 Echo is out of ADC range
19 Reserved
20 Missing normalisation tables
21 Number of samples in window is low
22 Number of samples in window is very low
23 Negative backscatter in resampled data
24 Possible interference from solar array
25 Reserved
26 Reserved
27 Reserved
28 Land
29 Water
30 Ascending pass
31 Descending pass
All 32 Missing
Flag Table 0-33-113 : IASI-NG general quality flags
Bit Meaning
1 General quality (0 if bits 2 to 11 are all 0, 1 otherwise)
2 Sounder radiometric
3 Sounder spectral
4 Imager geometric
5 Imager classification
6 Imager radiometric
7 General sounder
8 Presence of mathematical errors
9 TM data missing
10 TM data corrupt
11 TM integrity instrument
12-15 Reserved
All 16 Missing
Flag Table 0-33-114: Out of bounds check
Bit Meaning
1 Temperature profile
2 Water-vapour profile
3 Ozone profile
4 Surface skin temperature
5 Surface air temperature
6 Surface air dew point temperature
7 Surface pressure
8 Surface skin temperature error estimate (quality indicator)
9 Surface air temperature error estimate (quality indicator)
10 Surface air dew point temperature error estimate (quality indicator)
11 Surface pressure error estimate (quality indicator)
12 Surface emissivity coarse infrared spectrum
13 Surface emissivity microwave spectrum
14 CO2 profile
15 CH4 profile
16 N2O profile
All 17 Missing

Comments

No response

Requestor(s)

EUMETSAT : Simon Elliott Antoine MERLE

Stakeholder(s)

ECMWF WMO ECCODES

Publication(s)

New entries in : Table B New entries in : Table D New entries in : Flag Table New entries to existing flag table

Expected impact of change

MEDIUM

Collaborators

EUMETSAT scientists

References

No response

Validation

No response

antoinemerle commented 2 years ago

Dear @jbathegit

CURRENT STATE

As reminder on MWS sequence, you made an interesting remark on the following part of the sequence :

NEW PROPOSAL

we made a workaround and find a solution : would it be okay for you if we only request to create two new descriptors:

and replace the sequence then by :

Conclusion

The difference would be impact the current proposal :

Is this okay for you ?

Thanks a million in advance for your help

antoinemerle commented 2 years ago

Dear @marijanacrepulja

Current situation on MWI and ICI

code name unit scale ref width Mission
0-05-193 Sub-satellite latitude Degrees 4 -900000 21 ICI, MWI
0-06-194 Sub-satellite longitude Degrees 4 -1800000 22 ICI, MWI

For those two descriptors, you asked an interesting question :

what Sub-satellite latitude mean ?

after looking and ask scientist for it, I can tell you that the correct name should only be :

according to EUMETSAT scientists and definition :

Sub satellite Point is the point at which a line between the satellite and the center of the Earth. intersects the Earth's surface. The location of the point is expressed in terms of latitude and. longitude.

source https://kanchiuniv.ac.in/coursematerials/satcomm.pdf

Proposal :

Changing the name of :

Would it be better and more clear for you ? Thanks a million in advance

Antoine MERLE

jbathegit commented 2 years ago

Hello @antoinemerle

re: "Satellite channel band width", there's already an existing Table B descriptor 0-02-154 which you can use for this, and which in fact already has the same Hz | -8 | 0 | 26 specifications. So you don't need to propose a new F-X-Y number for this.

re: the proposed new 0-05-193 and 0-05-194, I have more often heard that point referred to as the "Satellite sub-point", rather than the "Sub-satellite point". However, there do appear to be ample references which use the latter term as well, so it's probably fine.

re: the proposed new 0-21-192 and 0-21-193, it's not clear what you mean by "low resolution" vs. "high resolution", given that both of these use the same Numeric | 4 | 0 | 14 specifications. But either way, there's already an existing Table B descriptor 0-21-166 for "Land Fraction" with Numeric | 3 | 0 | 10, so you could just use Table C operators to modify those specifications as needed, and therefore you shouldn't need to propose any new F-X-Y numbers for this.

re: the proposed new 0-07-195 and 0-07-196, I have the same question here as to what you mean by "low resolution" vs "high resolution", again given that both of these are proposed with the same specifications. And similarly, is there any way you could just use one of the existing height descriptors in Class 7 and, if needed, modify the specifications using Table C operators, rather than proposing new Table B descriptors? Or, if these are non-coordinate height values, you could also use the existing Table B descriptor 0-10-001 which has the exact same name as what you are proposing.

antoinemerle commented 2 years ago

Dear Jeff,

re: "Satellite channel band width", there's already an existing Table B descriptor 0-02-154 which you can use for this, and which in fact already has the same Hz | -8 | 0 | 26 specifications. So you don't need to propose a new F-X-Y number for this.

Indeed you are completely right here :
the final result would be then 1 02 004 - replicate the 2 descriptors 4 times 0 05 195 - band number 0 02 154 - Satellite channel bandwidth 0 05 195 - band number = missing ==> only request to create one new descriptor in table B: 0 05 195 TODO @antoinemerle update the sequence for MWS and edit the issue accordingly to that recomendation. also remove the extra entries from table B

re: the proposed new 0-05-193 and 0-05-194, I have more often heard that point referred to as the "Satellite sub-point", rather than the "Sub-satellite point". However, there do appear to be ample references which use the latter term as well, so it's probably fine.

Thanks for your time ==> great to hear TODO @antoinemerle update the table B

re: the proposed new 0-21-192 and 0-21-193, it's not clear what you mean by "low resolution" vs. "high resolution", given that both of these use the same Numeric | 4 | 0 | 14 specifications. But either way, there's already an existing Table B descriptor 0-21-166 for "Land Fraction" with Numeric | 3 | 0 | 10, so you could just use Table C operators to modify those specifications as needed, and therefore you shouldn't need to propose any new F-X-Y numbers for this. re: the proposed new 0-07-195 and 0-07-196, I have the same question here as to what you mean by "low resolution" vs "high resolution", again given that both of these are proposed with the same specifications. And similarly, is there any way you could just use one of the existing height descriptors in Class 7 and, if needed, modify the specifications using Table C operators, rather than proposing new Table B descriptors? Or, if these are non-coordinate height values, you could also use the existing Table B descriptor 0-10-001 which has the exact same name as what you are proposing.

According to us, it already exists some codes with "high resolution" in their name as the following code 0 07 071 Height (high resolution) , for us that justify the fact to have a descriptor for our needs. I had a discussion with scientists, Here is the explanation of the height vs low

the MWS processor gets a variable “lsmdem_search_resol” (LSM=Land Sea Mask, DEM= Digital Elevation Model). This variable has the dimension n_resol (that also surface_type and surface_elevation will have in addition to the nscans,nfov dimensions), i.e. the variable provides two values (in kilometres, referring to distances). For the surface type/ elevation determination, any LSM/DEM grid points within this distance “lsmdem_search_resol” (1st and 2nd value) to the MWS FOV are collected. Hence, for the large distance (i.e. radius) more grid points are collected than for the smaller distance. From each of the two grid point collections, the fraction of land grid points is determined and saved as surface_type of the current MWS FOV for a high resolution and low resolution (i.e. based on a smaller or larger radius for collecting the grid points). So, basically, the high/low resolution refers to the “allowed distance” in km of a LSM/DEM map grid point to the current FOV. “Allowed” meaning included in the determination process of the respective quantity for the current MWS FOV.

We still need a distinction between the low vs higher resolution for the final user. Nevertheless, I hear your comments, we can maybe organize a meeting to discuss it. Maybe it will be quicker to fix this problem / suggestions.

I really thank you for your time and your advices.

antoinemerle commented 2 years ago

EDIT on IASI L1C and L1D During our most recent interactions with our expert scientist in EUMETSAT, some small issues have come to light with the IASI-NG L1C and L1D BUFR representation. We have updated the specifications accordingly with @SimonElliottEUM

antoinemerle commented 2 years ago

EDIT 2 : Scientists ask to add the sub-centre. we added the code 3 01 005 to the IASI L1C and L1D template.

We added the following code to replace the two codes for identification of the originating center and sub-centre. Impacted sequences are those two sequences 3 10 197 (IASI-NG L1C) and 3 10 198 (IASI-NG L1D):

code name
3 01 005 Originating centre/sub-centre
antoinemerle commented 2 years ago

Hi @jbathegit

I follow your recommendations and I implemented the following changes with Simon Elliott

MWS : related to the resolution problem

We won't create 4 new descriptors anymore, but only one new for Radius resolution : 0 05 0196

from : code description
0 21 192 Land fraction - low resolution
0 07 195 Height of land surface - low resolution
0 21 193 Land fraction - high resolution
0 07 196 Height of land surface - high resolution
to : code description
1 07 002 Repeat 7 descriptors 2 times
0 05 196 Radius resolution
2 01 132 Change width (+4)
2 02 129 Change scale (+1 )
0 21 166 Land fraction
2 02 000 Change scale (reset to 3)
2 01 000 Change width (reset to 10)

MWS : related to the satellite channel band width

We don't create 4 new descriptors but only one new for Band number : 0 05 195 From : code description
0 40 192 Satellite channel band width (first band)
0 40 193 Satellite channel band width (second band)
0 40 194 Satellite channel band width (third band)
0 40 195 Satellite channel band width (fourth band)
to : code description
1 02 004 Repeat 2 descriptors 4 times
0 05 195 Band number
0 02 154 Satellite channel band width
amilan17 commented 2 years ago

https://github.com/wmo-im/CCT/wiki/Teleconference-1-and-2.11.2022 notes:

antoinemerle commented 2 years ago

EDIT : update the zip file with the definitions files

Dear @amilan17 , we updated the table B with the final code (chosen by us). please tell me if this is okay for you and for the secretary. We also added an element description column, as requested in your last comments.

Dear @marijanacrepulja and @SibylleK, Please find attached :

Thanks for your help and time,

I am at your disposal for any questions .

bufr_samples_and_tables.zip

marijanacrepulja commented 2 years ago

Dear @antoinemerle,

Thank you for providing the samples.

I had a look at the zip file and can only find sample BUFR files but not corresponding Eccodes tables.

As part of the validation process, all proposed BUFR descriptors, templates and code tables have to be entered into git hub branch. You can find the branch on the right hand side of the issue definition under Development. Please note the master table should be set to 40 and local table to 0 in the sample files. Because, 40 is the next version of the tables that will contain your proposed entries.The idea of the validation process is that we can download tables from the git hub, and decode provided samples.

I understand it is a lot of work, but we need to validate samples using BUFR tables from git hub, as Anna will be merging it into the table release's final branch.

Many thanks!

antoinemerle commented 2 years ago

Dear @marijanacrepulja ,

I just updated the zip file with the definitions tables ( you can find it above in previous comment, I am sorry for the mistake).

As you requested, I tried to generate my products with table master V40 (and V39) , but the latest Eccodes V2.27.0 (delivery date August 23) does not allow me to create a BUFR file with any Master table version above 38 (included).

When trying to create a product I got the following error:

ecCodes assertion failed: `included_fname' in /tmp/eccodes-2.27.0-Source/src/grib_parse_utils.c:639

According to Eccodes documentation :

The latest time BUFR received a WMO master table version update is with the release Eccodes 2.26.0 ( source )

marijanacrepulja commented 2 years ago

Dear @antoinemerle,

Thank you for updating the zip file.

To create samples with master table version 40. You should create locally mkdir -r mydefinition/bufr/tables/0/wmo/40/

For the purpose of validation, all new entries have to be in git hub branch, so any software for decoding/encoding BUFR can handle the samples provided.

Hope this helps.

Many thanks, Marijana

shahramn commented 2 years ago

It is always safer to set the FULL PATH of your own definitions in the environment variable Something like export ECCODES_DEFINITION_PATH=$PWD/mydefinition

antoinemerle commented 2 years ago

Hi @shahramn, I fully agree with you and this is actually the case in my configuration :

echo $ECCODES_DEFINITION_PATH
/usr/src/eccodes/share/eccodes/definitions/

echo $ECCODES_DIR
/usr/src/eccodes/

The error I shared was rendered at runtime, while encoding the BUFR file, maybe when running the script, python is not picking the right one. I will investigate.

Hi @marijanacrepulja Thanks for the input and help with version 40 :

mkdir -r mydefinition/bufr/tables/0/wmo/40/ copy latest element.table sequence.def and all codetables into the mydefinition/bufr/tables/0/wmo/40/ update element.table, sequence.def and codetables with new definitions

I definitely will do it today.

One additional question, I can commit all the changes in the same branch ? Right ?

Thanks

marijanacrepulja commented 2 years ago

Hi @antoinemerle

Yes, having all the changes in one branch is convenient.

Many thanks!

amilan17 commented 2 years ago

https://github.com/wmo-im/CCT/wiki/Teleconference-21.11.2022 notes: @jbathegit will be able to review sometime next week. @antoinemerle is updating the branch @marijanacrepulja and @SibylleK will validate

amilan17 commented 2 years ago

Dear @amilan17 , we updated the table B with the final code (chosen by us). please tell me if this is okay for you and for the secretary.

@antoinemerle -- looks good so far.

amilan17 commented 2 years ago

@antoinemerle -- can you add acronyms to this CSV? acronyms.csv

amilan17 commented 2 years ago

@antoinemerle -- can you add acronyms to this CSV? acronyms.csv

antoinemerle commented 2 years ago

@antoinemerle -- can you add acronyms to this CSV? acronyms.csv

Dear @amilan17 what acronyms ? ( Can you give me an example just for me to know.) @amilan17 : You've noticed that I pushed my changes in my branch, efucile already made some changes : I take a note that I will have to update my script to update also the XML tables in the next delivery.

Dear @marijanacrepulja @SibylleK : I just uploaded the table 40 and the corresponding products here:

Please let me know if I missed something wmo_delivery_v40.zip

Thank you all for your time and dedication.

Antoine

jbathegit commented 2 years ago

https://github.com/wmo-im/CCT/wiki/Teleconference-21.11.2022 notes: @jbathegit will be able to review sometime next week

I have no further comments or concerns - thanks!

amilan17 commented 1 year ago

@antoinemerle acronyms like: MWI, ICI, SCA, IAS 1, MWS...

antoinemerle commented 1 year ago

Dear @amilan17 I just commit the changes : SHA code : b71910af37c17ddefd60f3b6c56f980b7cd2b732 link of the commit : https://github.com/wmo-im/BUFR4/commit/b71910af37c17ddefd60f3b6c56f980b7cd2b732

SibylleK commented 1 year ago

Dear @antoinemerle, I just wanted to validate the examples using the branch for this issue, but for the validation the Table D entries are also needed in the branch. Is it possible for you to also add the sequences to the branch?

antoinemerle commented 1 year ago

Dear @SibylleK , It is done. : fc9199027b8e452429893e75a4d7b3f7a740165a click here to see the commit I already did the changes but forgot to run the git commit. All the product have been added in the BUFR_TableD_en_10.csv now It should be okay , I am double-checking.

@efucile I assume we need once more to generate the corresponding XML and txt file from that BUFR_TableD_en_10.csv

amilan17 commented 1 year ago

@antoinemerle - thank you for updating the branch. The XML and TXT files are generated automatically and Enrico's handle is just part of the script for legacy reasons.

antoinemerle commented 1 year ago

@antoinemerle - thank you for updating the branch. The XML and TXT files are generated automatically and Enrico's handle is just part of the script for legacy reasons.

HI @amilan17 I had to change the sequence for MWS. as the commit title is saying :

Following another check from the EUMETSAT scientists, we needed to change the resolution from 100MHz to 1MHz for the Satellite Channel band width. Meaning we had to add 2 to the scale and we had to change the sequence for MWS.

image

May you run again the the script for the XML and TXT Files ?

Thanks a million

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference-12-and-13.12.2022 notes:

sibylle was unable to validate the large number of subsets; @antoinemerle can provide a smaller product for testing.

antoinemerle commented 1 year ago

@SibylleK , please find attached the MWI and ICI with "only" 5,576 subsets representing 4 scan lines. Tell me if this is better for you :) (I will provide tinier ones for SCA two..) MWI AND ICI.zip

Question for you also, because Anna did not write it here but here is the list of changes still to be applied : can you confirm and/or correct me :

amilan17 commented 1 year ago

@antoinemerle -- I'm in the middle of a deep consistency analysis of the CSVs and and it's easier for me to update the branch directly for editorial things like you mentioned above. I will probably have an update of the branch by end of the day with some follow up questions. Do you mind waiting on any further updates until then?

antoinemerle commented 1 year ago

@amilan17 no worries at all on my side at all ! 👍 Thanks for letting me know

amilan17 commented 1 year ago

@antoinemerle I completed the consistency review and made some basic editorial changes for you to approve of in PR #141.

 I have some other suggestions and questions. 

  1. Table B 21 already has a descriptor for ASCAT sigma-0 usability (021159), but the data width is 2 bits instead of 3. Is it possible to use the existing descriptor instead of creating another one with the same name?
  2. Flag tables 
    1. some of the ICI and MWI flag tables are almost identical -- does it make sense to combine them into one table? 
  3. BUFR Table D 10
    1. "008098, Source of temperature measurement, Cancel significance (15)"
      1. I wonder if this is correct, because code figure 15 is set to "Missing" in table 008098.
    2. "10,Vertical sounding sequences (satellite data),310081,(Microwave Imager Data (level 1B)),,117003,Replicate 17 descriptors 3 times 333,,,,Operational"
      1. Is "333" in the element name a typo?  
    3. The element names of data description operators should be the same as the operator name in BUFR Table C and the details go into the element description column. Can you update these?
      1. 201YYY == Change data width
      2. 202YYY == Change scale
      3. 207YYY == Increase scale, reference value and data width
      4. for example: "20100, Cancel change of width, " should be "20100, Change data width, Cancel"
SibylleK commented 1 year ago

@amilan17 Thank you very much for the editorial changes which already include some changes of the units!

There are still some inconsistencies regarding the units in 033112 (deg with a capital D), 003029, 003030, 008098 and 021026 (Code table with a capital T) and the new Flag tables in BUFRCREX_TableB_en_33.csv which also have a capital T.

@antoinemerle the new MWI-BUFR examples still have 27880 subsets and the version number is set to 38:-( Could you check? Regarding the ICI, the BUFR also have the same number of subsets and version 38, but in this case I got another error ("Errormessage: End of descriptors in repetition list"), which indicate that the sequence does not work as it is in the branch. It seems that the last 23 entries of sequence 310080 are not included in the descriptor of the branch.

amilan17 commented 1 year ago

There are still some inconsistencies regarding the units in 033112 (deg with a capital D), 003029, 003030, 008098 and 021026 (Code table with a capital T) and the new Flag tables in BUFRCREX_TableB_en_33.csv which also have a capital T.

@SibylleK Thanks for letting me know. They should be fixed now.

antoinemerle commented 1 year ago

Dear @SibylleK thanks for the feedback.

Please find the correct files attached : MWI_and_ICI.zip

I fixed the issue, It was due to a wrong copy/past (I missed the last 23 lines). thanks a lot for spotting that issue.

amilan17 commented 1 year ago

Some editorial updates are in this PR: #141

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference-10.01.2023 notes:

 after PR approval @amilan17 will merge PR, @marijanacrepulja still reviewing content, @SibylleK will review branch and can put output from examples in the SharePoint, @antoinemerle can post the NetCDF in the SharePoint and compare output to the NetCDF.

Teams SharePoint directory: https://wmoomm.sharepoint.com/:f:/s/wmocpdb/EnqK9Y2bfGxPnXNM_yrOq5gBJrfprspEPKpXr7YVIivSbQ?e=lvTWvU

(post to 'documentation' folder)

antoinemerle commented 1 year ago

Hi @amilan17


Table B 21 already has a descriptor for ASCAT sigma-0 usability (021159), but the data width is 2 bits instead of 3. Is it possible to use the existing descriptor instead of creating another one with the same name?

MERLE : For that case the code table are a bit different and our new one is more accurate one (according to me)


Flag tables : some of the ICI and MWI flag tables are almost identical -- does it make sense to combine them into one table? 

MERLE : Almost identical but not identical, Those are still different instruments with different meanings.


BUFR Table D 10 : "008098, Source of temperature measurement, Cancel significance (15)" I wonder if this is correct, because code figure 15 is set to "Missing" in table 008098.

MERLE : 15 means missing which means also cancel. I mean this is the way we did it the past (I can confirm this info with Simon)


"10,Vertical sounding sequences (satellite data),310081,(Microwave Imager Data (level 1B)),,117003,Replicate 17 descriptors 3 times 333,,,,Operational" Is "333" in the element name a typo?

MERLE : It was a typo, I already corrected after your message


The element names of data description operators should be the same as the operator name in BUFR Table C and the details go into the element description column. Can you update these? 201YYY == Change data width 202YYY == Change scale 207YYY == Increase scale, reference value and data width for example: "20100, Cancel change of width, " should be "20100, Change data width, Cancel"

MERLE : for that particular comment, I did not understand it @amilan17 : please can you give me more information, I am willing to correct it

amilan17 commented 1 year ago

@antoinemerle 

Thank you for your thorough feedback. The FXY sequences that begin with 201, 202 and 207 have predefined element names. The requested change is purely editorial, makes the text consistent and doesn't change the meaning. Below is a table with some detailed examples with the crossed out text replaced by the following line. You can also do a simple filter on the tables in the GitHub browser wiindow to see how these types of fields are written for other BUFR sequences. I hope this makes more sense, but I'm happy to explain more on Teams if you would like. 

FXY2 ElementName_en ElementDescription_en
~201000~ ~Cancel change of width~  
201000 Change data width Cancel
~202000~ ~Cancel change of scale~  
202000 Change scale Cancel
~207003~ ~Increase scale, reference value and data width (+3)~  
207003 Increase scale, reference value and data width  
antoinemerle commented 1 year ago

Dear @amilan17 , as promised :

Here are the new product for MWS and SCA in tinier part. Each file is containing 1 BUFR message of 10 scans, so each file is contains 10 scans. I also attached the netCDF used for encoding as ref.

PS : I tried to upload but it did not work with all the files so please at the moment find : for_WMO_lighter.zip

PS 2 : I updated the text for the cancel description : 3c113e2

amilan17 commented 1 year ago

@marijanacrepulja @SibylleK Will you be able to validate the sample data before Friday? Thanks!

SibylleK commented 1 year ago

The provided BUFR examples could be read by the DWD BUFR software. I uploaded the BUFR and the DWDoutput in the SharePoint documents under a issue136-NewEntries4EPS-SG folder.

I can't find the netCDFs. For comparison and check of the output some other outputs are needed. Maybe @marijanacrepulja is able to compare the DWD outputs with the output of eccodes, which would be sufficient for the technical validation. I can't judge, whether the values are correct. Maybe I get some comments of my colleagues.

Regarding some values of Flag tables, the encoding seems to be not correct, as the last bit is often set, which should not be. But this is not an obstacle for the validation.

amilan17 commented 1 year ago

@SibylleK @antoinemerle I made some editorial changes to element names and descriptions. Can you please double check that I didn't lose/change any meanings? (see just commit above)

antoinemerle commented 1 year ago

Greetings @amilan17 and @SibylleK ,

Regarding amilan17's commit : I have reviewed the necessary commit and made some minor comments in the commits section.

Regarding the test data: You should now find the folder "MWS_and_SCA_and_the_netCDF_ref.zip" in the SharePoint. This folder contains the following:

Please confirm that you have received and have access to all necessary files. If there are any issues or missing documents, please let me know.

Thanks for your support , let me know if you need anything else

marijanacrepulja commented 1 year ago

@amilan17 @antoinemerle @SibylleK

I was able to decode all BUFR examples, and when I compared the results to those provided by Sibylle, the values were identical. I can't examine the messages in depth because there are no corresponding netCDF files for each BUFR sample. As a result, I believe the proposed new descriptors and sequences are technically valid.

amilan17 commented 1 year ago

@marijanacrepulja Thank you.

antoinemerle commented 1 year ago

@marijanacrepulja Thanks a million for this last review.

Also : @amilan17 , @SibylleK and @marijanacrepulja Thanks for the hard work and all the time everyone invested in the review of this long request. Happy to help in any case. 🚀

amilan17 commented 1 year ago

@antoinemerle Can you please update this branch with the edit needed? When ready please open a PR to merge this branch into FT2023-1 branch.

https://github.com/wmo-im/BUFR4/tree/issue136-update2

marijanacrepulja commented 1 year ago

@amilan17 @SimonElliottEUM @antoinemerle

In the BUFR Template 310082 there is a difference between a branch and a proposal related to replications

Proposal

0 33 109 | MWS overall quality flags |   1 16 024 | Repeat 16 descriptors 24 times |   0 05 042 | Channel number |   1 02 004 | Repeat 2 descriptors 4 times |   0 05 079 | Band number

Git branch 033109,MWS overall quality flags 18024,Repeat 18 descriptors 24 times, 005042,Channel number 104004,Repeat 4 descriptors 4 times 005079,Band number

@SimonElliottEUM @antoinemerle Could you please confirm which of the entries is correct?

Many thanks

antoinemerle commented 1 year ago

Hi @marijanacrepulja ,

The correct one is well the git branch. No action is needed on the code or repo

I should have updated the proposal when we changed the git branch, I will update the proposal according to what we pushed. I apologize for the inconvenience that it brought.

So it is completely correct is as it is in the branch :

10,Vertical sounding sequences (satellite data),310082,(Microwave Sounder Data (level 1B)),,118024,Replicate 18 descriptors 24 times,,,,Operational
10,Vertical sounding sequences (satellite data),310082,(Microwave Sounder Data (level 1B)),,005042,Channel number,,,,Operational
10,Vertical sounding sequences (satellite data),310082,(Microwave Sounder Data (level 1B)),,104004,Replicate 4 descriptors 4 times,,,,Operational
10,Vertical sounding sequences (satellite data),310082,(Microwave Sounder Data (level 1B)),,005079,Band number,,,,Operational
10,Vertical sounding sequences (satellite data),310082,(Microwave Sounder Data (level 1B)),,202130,Change scale,Add 2 to descriptor scale,,,Operational
10,Vertical sounding sequences (satellite data),310082,(Microwave Sounder Data (level 1B)),,002154,Satellite channel band width,,,,Operational
10,Vertical sounding sequences (satellite data),310082,(Microwave Sounder Data (level 1B)),,202000,Change scale,Cancel,,,Operational

a bit of details

The very first initial proposal for MWS on this particular part of the sequence was

code description
1 17 024 Repeat 17 descriptors 24 times
0 05 042 Channel number
0 40 192 Satellite channel band width (first band)
0 40 193 Satellite channel band width (second band)
0 40 194 Satellite channel band width (third band)
0 40 195 Satellite channel band width (fourth band)

But during the :

we made some changes to this part and we moved to the following sequence :

code descr
1 18 024 Repeat 18 descriptors 24 times
0 05 042 Channel number
1 04 004 Repeat 4 descriptors 4 times
0 05 079 Band number
2 02 130 Change scale (from -8 to -6)
0 02 154 Satellite channel band width
2 02 000 Change scale (reset to -8)

the following was an intermediate state :

0 33 109 | MWS overall quality flags |
1 16 024 | Repeat 16 descriptors 24 times |
0 05 042 | Channel number |
1 02 004 | Repeat 2 descriptors 4 times |
0 05 079 | Band number