wmo-im / BUFR4

BUFR edition 4
MIT License
27 stars 9 forks source link

Proposal for new BUFR table entries for EPS-SG (EUMETSAT) : IASI-NG Level 2 and VII Level 2 #161

Closed antoinemerle closed 11 months ago

antoinemerle commented 1 year ago

Initial request

This document proposes new BUFR entries for the following L2 products :

Product Presented to TT-TDCF group reviewed/corrected new code fixed code pushed Data sample provided Approved READY
Common IASI-NG L2 info R
IASI-NG L2 TWV R
IASI-NG L2 CLD R
IASI-NG L2 SFC R
IASI-NG L2 GHG still under investigation about pressure R
IASI-NG L2 CO_ R
IASI-NG L2 NAC and 03 R
IASI-NG L2 SO2 still under investigation about pressure R
VII L2 CLD R
VII L2 CTP R
VII L2 OCA R
VII L2 WVV and WVI R

This document proposes new BUFR entries for the following L2 products :

Amendment details

New entries in Table B :

Add the following elements to BUFR Table B :

F X Y Element name Unit Scale Reference value Bit width Instrument
0 12 083 Temperature difference between observed and computed K 2 -10000 16 IASI-NG L2
0 10 073 Height of plume m 2 0 21 IASI-NG L2
0 08 071 Type of cost estimation Code table 0 0 3 IASI-NG L2
0 12 169 Brightness temperature difference K 3 -100000 18 IASI-NG L2
0 15 007 Estimated amount of chemical constituent DU 2 0 16 IASI-NG L2
0 33 116 First guess in bounds check Flag Table 0 0 17 IASI-NG L2
0 33 117 In bounds check Flag Table 0 0 17 IASI-NG L2
0 40 080 Cloud test results for IASI-NG level 2 Flag Table 0 0 5 IASI-NG L2
0 40 075 Retrieval error covariance matrix for humidity in principal component domain Numeric 0 0 21 IASI-NG L2
0 40 076 Retrieval error covariance matrix for temperature in principal component domain Numeric 0 0 21 IASI-NG L2
0 25 144 Number of visual channels used in processing Numeric 0 0 6 VII-02-OCA
0 25 145 Number of infrared channels used in processing Numeric 0 0 6 VII-02-OCA
0 20 139 Final cloud model assumption Code table 0 0 4 VII-02-OCA
0 13 094 log 10 of Cloud optical thickness Numeric 3 -2000 14 VII-02-CTP,VII-02-OCA
0 33 118 Relative error percentage % 2 -10000 15 VII-02-WVV, VII-02-WVI
0 33 119 Forecast error percentage % 0 -100 8 VII-02-WVV, VII-02-WVI

New entries in existing code and flag table

  Code Table 0 08 072 :
Action Value New meaning
Add 5 Ash
Update 6 Reserved
  Code Table 0 08 029: Surface type
Add 19 Forests
Add 20 Shrublands
Add 21 Savanna
Add 22 Desert
Add 23 Grasslands
Add 24 Wetlands
Add 25 Snow/Ice
Add 26 Coast
Update 27 - 254 Reserved
  Code Table 0 33 028: Snapshot overall quality
Action Value New meaning
Add 5 General degradation
Update 6 Reserved

New flag tables and related code

Code Table 0 08 071 : Type of cost estimation
Value Meaning
0 First guess
1 Y vector
2 X vector
3 to 6 Reserved
7 Missing
Flag Table 0 33 116 : First guess in bounds check
Bit Meaning
1 Temperature profile
2 Water-vapour profile
3 Ozone profile
4 Surface skin temperature
5 Surface air temperature
6 Surface air dew point temperature
7 Surface pressure
8 Surface skin temperature error estimate(quality indicator)
9 Surface air temperature error estimate(quality indicator)
10 Surface air dew point temperature error estimate (quality indicator)
11 Surface pressure error estimate (quality indicator)
12 Surface emissivity coarse infrared spectrum
13 Surface emissivity microwave spectrum
14 CO2 profile
15 CH4 profile
16 N2O profile
All 17 Missing
Flag Table 0 33 117: In bounds check
Bit Meaning
1 Temperature profile
2 Water-vapour profile
3 Ozone profile
4 Surface skin temperature
5 Surface air temperature
6 Surface air dew point temperature
7 Surface pressure
8 Surface skin temperature error estimate(quality indicator)
9 Surface air temperature error estimate(quality indicator)
10 Surface air dew point temperature error estimate (quality indicator)
11 Surface pressure error estimate (quality indicator)
12 Surface emissivity coarse infrared spectrum
13 Surface emissivity microwave spectrum
14 CO2 profile
15 CH4 profile
16 N2O profile
All 17 Missing
Flag Table 0 40 080: Cloud test results for IASI-NG level 2
Bit Meaning
1 Thin cirrus detection executed
2 Thin cirrus detection indicates a cloud
3 Aerosol dust detection executed
4 Aerosol dust detected
All 5 Missing
Code Table 0 20 139 : Final cloud model assumption for VII L2
Value Meaning
0 Clear
1 Single layer water
2 Single layer ice
3 2-layers (water/ice)
4 single layer ash
5 2-layers ash
6-14 Reserved
15 Missing

New table D entries

New element Element name
3 10 087 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (TWV, CLD, SFC, GHG)
3 10 088 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 temperature and water vapour profiles (TWV)
3 10 089 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 cloud mask cloud detection (CLD)
3 10 090 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 surface parameters (SFC)
3 10 091 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 total columns of nitrous oxide, methane and carbon dioxide (GHG)
3 10 092 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (IASI NG L2 CO_, NAC, O3, SO2)
3 10 093 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 carbon monoxide total column, carbon monoxide profile (CO_)
3 10 094 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 nitric acid total column, nitric acid profile (NAC) and level 2 ozone total column, ozone profile (O3_)
3 10 095 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2)
3 10 096 Metop-SG Visible Infrared Imager (VII), level 2 common part for VII L2 CLD, CTP, OCA, WVI, WVV
3 10 097 Metop-SG Visible Infrared Imager (VII), level 2 cloud cloud detection (CLD)
3 10 098 Metop-SG Visible Infrared Imager (VII), level 2 Cloud top pressire (CTP)
3 10 099 Metop-SG Visible Infrared Imager (VII), level 2 Optimal Cloud Analysis (OCA)
3 10 100 Metop-SG Visible Infrared Imager (VII), level 2 Total precipitable water IR and VIS (WVI/WVV)

IASI L2 - Common part for TWV,CLD, SFC, GHG from EUM-PRS

Add the following sequence 3 10 087 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 PRS common part for TWV,CLD,SFC,GHG

Here's the formatted table in Markdown:

Table references Element name Note
0 01 007 Satellite identifier
0 01 031 Identification of originating/generating centre
0 02 019 Satellite instruments
0 02 020 Satellite classification
0 04 001 Year
0 04 002 Month
0 04 003 Day
0 04 004 Hour
0 04 005 Minute
2 02 131 Change scale
2 01 138 Change data width
0 04 006 Second
2 01 000 Change data width
2 02 000 Change scale
0 05 001 Latitude (high accuracy)
0 06 001 Longitude (high accuracy)
0 07 024 Satellite zenith angle
0 05 021 Bearing or azimuth
0 07 025 Solar zenith angle
0 05 022 Solar azimuth
0 05 043 Field of view number
0 05 040 Orbit number
2 01 133 Change data width
0 05 041 Scan line number
2 01 000 Change data width
2 02 126 Change scale
0 07 001 Height of station
2 02 000 Change scale
0 40 052 Indication of super-adiabatic and super-saturation in final retrieval
0 40 047 Validation flag for IASI or IASI-NG level 1 product
0 08 065 Sun-glint indicator
2 04 004 Add associated field with 4 bits Adding 4 bits to give quality information on the data to be processed
0 31 021 Associated field significance 9 for specifying for each of the incoming variable the status of the data retrieved.
0 02 019 Satellite instruments For MetImage
0 02 019 Satellite instruments For the embedded MWS
0 01 030 Numerical model identifier For the numerical model ECMWF
2 04 000 Cancel associated field
0 02 019 Satellite instruments Back to instrument IASI-NG
0 40 046 Cloudiness summary
0 33 116 First guess in bounds check First guess means retrieved with PWLR method and It checks that the values are within the expected range for the correspondent geophysical parameter
0 33 117 In bounds check In bounds check means retrieved with OEM. It checks that the values are within the expected range for the correspondent geophysical parameter
0 40 080 Cloud test results for IASI-NG level 2
0 40 051 Convergence of the iterative retrieval
0 08 003 Vertical significance (satellite observations) Set to 0=surface
0 21 166 Land fraction
0 22 046 Sea ice fraction
0 10 001 Height
0 08 023 First-order statistics Standard deviation
0 10 001 Height
0 08 023 First-order statistics Cancel the first-order-statistics
0 08 003 Vertical significance (satellite observations) Cancel the vertical significance

IASI L2 - TWV

Details on 3 10 088 Add the following sequence 3 10 88: Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 temperature and water vapour profiles (TWV) Here's the formatted table in Markdown: | Table references | Element name | Note | | --- | --- | --- | 3 10 087|IASI NG L2 common information part|The common information shared for TWV, CLD, SFC and GHG 0 40 053|Number of iterations used for retrieval| 1 02 003|Replication of 2 elements 3 times| 0 08 071|Type of cost estimation| 0 25 149|Optimal estimation cost| 1 06 000|Delayed replication of 6 descriptor| 0 31 000|Short delayed replication factor| 1 01 000|Delayed replication of 1 descriptor| 0 31 002|Extended delayed descriptor replication factor| 0 40 076|Retrieval error covariance matrix for temperature in principal component domain| 1 01 000|Delayed replication of 1 descriptor| 0 31 002|Extended delayed descriptor replication factor| 0 40 075|Retrieval error covariance matrix for humidity in principal component domain| 0 40 067|Quality indicator for atmospheric temperature| 0 40 066|Quality indicator for atmospheric water vapour| 2 02 126|Change scale| 0 13 098|Integrated water vapour (column)| 2 02 000|Cancel| 1 07 101|Replication of 7 elements 101 times|For 101 pressure layer 2 02 131|Change scale| 2 01 138|Change width| 0 07 004|Pressure|At a given pressure 2 01 000|Cancel| 2 02 000|Cancel| 0 12 101|Air temperature|Air temperature at given pressure 0 13 001|Specific humidity|Humidity at given pressure > note for the short delayed replication Basically we will have either or not a matrice of error for temperature and humidity for each point. While we are reading the error_data_index that is describing the index of the point : If we have 255 in the value of the error_data_index, that means there is no errors, we will put 0 for the shor delayed replication factor (meaning we skip the other descriptors bellow) > if we a number (corresponding to the current index) we will put 1 and then the 6 following descriptors will be replicated meaning there is actually some error for that particular point

IASI-NG L2 CLD

Details on 3 10 89 Add the following sequence `3 10 089` : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 cloud detection (CLD) Here's the formatted table in Markdown: | Table references | Element name | Note | | --- | --- | --- | 3 10 087|IASI NG L2 common information part|The common information shared for TWV, CLD, SFC and GHG 0 12 083|Temperature difference between observed and computed|Difference in Kelvin float number between the observed temperature and the computed temperature 0 08 043|Atmospheric chemical or physical constituent type|Set to Dust 0 40 044|Dust index| 0 08 043|Atmospheric chemical or physical constituent type|Cancel Dust 0 20 056|Cloud phase|Set to 1 for liquid 0 13 109|Ice/liquid water path|Liquid mass 0 20 056|Cloud phase|Set to 2 for ice 0 13 109|Ice/liquid water path|Ice mass 0 08 003|Vertical significance (satellite observations)|Set vertical significance to cloud top 1 09 002|Replication of 9 descriptors 2 times|Loop other two layers of clouds (one at a first specific pressure and a second one at a second specific pressure) 0 20 056|Cloud phase|Describe the physical state of the cloud 0 20 081|Cloud amount in segment|Fractional cloud cover in percentage 2 02 131|Change scale| 2 01 138|Change width| 0 20 016|Pressure at top of cloud|Pressure at cloud top 2 01 000|Cancel| 2 02 000|Cancel| 0 12 101|Air temperature|Air temperature at cloud top 0 20 131|Effective radius of cloud hydrometeors|Cloud drop effective radius at cloud top

IASI-NG L2 SFC

Details on 3 10 090 Add the following sequence `3 10 090` : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 surface parameters (SFC) Here's the generated Markdown for the table: | Table references | Element name | Note | | --- | --- | --- | 3 10 087|IASI NG L2 common information part|The common information shared for TWV, CLD, SFC and GHG 0 08 003|Vertical significance (satellite observations)|Set Vertical significance to surface 0 40 067|Quality indicator for atmospheric temperature| 0 40 066|Quality indicator for atmospheric water vapour| 1 02 012|Replication of 02 descriptors 12 times|12 channels 0 05 042|Channel number|Number of channels 0 14 050|Emissivity|Emissivity at surface 0 12 001|Temperature / air temperature|Air temperature at surface 2 02 131|Change scale| 2 01 138|Change width| 0 10 004|Pressure|Pressure at surface 2 01 000|Cancel| 2 02 000|Cancel| 0 08 003|Vertical significance (satellite observations)|Cancel Vertical Significance 0 12 083|Temperature difference between observed and computed|Difference between the observed temperature and the computed temperature 0 11 012|Wind speed at 10m| 0 08 043|Atmospheric chemical or physical constituent type|Set Atmospheric chemical to Dust 0 40 044|Dust index| 0 08 043|Atmospheric chemical or physical constituent type|Cancel Dust

IASI-NG L2 GHG

Details on 3 10 091 Add the following sequence `3 10 091` : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 total columns of nitrous oxide, methane and carbon dioxide (GHG) | Table references | Element name | Note | | --- | --- | --- | 3 10 087|IASI NG L2 common information part|The common information shared for TWV, CLD, SFC and GHG 1 04 003|Replicate 4 descriptors 3 times |Loop other the 3 different chemical constituent type 0 08 043|Atmospheric chemical or physical constituent type | N20,CH4 and CO2 1 02 007|Replicate 2 descriptors 7 times| 0 07 004|Pressure |At a given pressure 0 15 021|Integrated mass density|

IASI L2 - Common part for CO_, NAC, O3, SO2 from EUM-PRS

Add the following sequence 3 10 092 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (IASI NG L2 CO_, NAC, O3, SO2) Table references Element name note
0 01 007 Satellite identifier
0 25 060 Software identification
0 01 031 Identification of originating/generating centre
0 02 019 Satellite instruments
0 02 020 Satellite classification
0 04 001 Year
0 04 002 Month
0 04 003 Day
0 04 004 Hour
0 04 005 Minute
2 02 131 Change scale
2 01 138 Change data width
0 04 006 Second
2 01 000 Change data width
2 02 000 Change scale
0 05 001 Latitude (high accuracy) Geocentric latitude at sounder pixel centre
0 06 001 Longitude (high accuracy) Geocentric longitude at sounder pixel centre
0 07 024 Satellite zenith angle Measurement zenith angle at sounder pixel centre
0 05 021 Bearing or azimuth Measurement azimuth angle at sounder pixel centre
0 07 025 Solar zenith angle Solar zenith angle at sounder pixel centre
0 05 022 Solar azimuth Solar azimuth angle at sounder pixel centre
0 05 043 Field of view number Field of view number
0 05 040 Orbit number Orbit number
2 01 133 Change data width
0 05 041 Scan line number Scan line number
2 01 000 Change data width
2 02 126 Change scale
0 10 007 Height Altitude of surface
2 02 000 Change scale

IASI-NG L2 CO_

Details on 3 10 093 Add the following sequence `3 10 093` : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 carbon monoxide total column, carbon monoxide profile (CO_) Table references | Element name | note | -- | -- | -- | 3 10 092|IASI NG L2 common information part|The common information shared for CO, NAC, O3 and SO2. 0 08 046|Atmospheric chemical or physical constituent type|4 Carbon monoxide CO 0 15 021|Integrated mass density|Mass content of Carbon monoxide 0 40 056|General retrieval quality|General retrieval quality flag 0 40 058|Number of vectors describing the char. matrices|Number of vectors describing the characterization matrices 0 40 059|Number of layers actually retrieved|Number of layers actually retrieved 0 40 060|Number of profiles retrieved in scanline|Number of for carbon monoxide profiles retrieved in scanline 0 40 057|IASI level 2 retrieval flags|IASI2 retrieval flag containing the 040054 and 04055 flag 1 03 019|Repeat 3 descriptors 19 times|Loop other 19 column 0 40 061|Air partial columns on each retrieved layer|molecules/cm 0 40 062|A-priori partial columns on each retrieved layer|Molecules/cm 0 40 063|Scaling vector multiplying the a priori vector in order to define the retrieved vector|Scaling vector multiplying the a priori CO vector in order to define the retrieved CO vector. 1 01 190|Repeat1 descriptor 190 times | 0 40 065|Main eigenvectors of the sensitivity matrix|Main eigenvectors of the sensitivity matrix

IASI-NG L2 NAC and IASI-NG L2 O3

Details on 3 10 094 Add the following sequence `3 10 094` : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 nitric acid total column, nitric acid profile (NAC) and level 2 ozone total column, ozone profile (O3_) Table references | Element name | note | -- | -- | -- | 3 10 092|IASI NG L2 common information part|The common information shared for CO, NAC, O3 and SO2. 0 08 046|Atmospheric chemical or physical constituent type|Here code for ozone(O3) or nitric acid (HNO3) 0 15 021|Integrated mass density|Mass content of nitric acid or of ozone 0 40 056|General retrieval quality flag|General retrieval quality flag 0 40 058|Number of vectors describing the char. matrices|Number of vectors describing the characterization matrices 0 40 059|Number of layers actually retrieved|Number of layers actually retrieved 0 40 060|Number of profiles retrieved in scanline|Number of HNO3 or O3 profiles retrieved in scanline 0 40 057|IASI level 2 retrieval flags|IASI 2 retrieval flag containing the 040054 and 04055 flag 1 03 041|Repeat 3 descriptor 41 times|Loop other each layer 0 40 061|Air partial columns on each retrieved layer|Air partial columns on each retrieved layer molecules / cm2 0 40 062|A-priori partial columns for each retrieved layer|A-priori partial columns for HNO3 or O3 in each retrieved layer 0 40 063|Scaling vector multiplying the a priori vector in order to define the retrieved vector|Scaling vector multiplying the a priori HNO3 or O3 vector in order to define the retrieved HNO3 or O3 vector. 1 02 021|Repeat 2 descriptor 21 times|Layer repeat 861 1 01 041|Repeat 1 descriptor 41 times| 0 40 065|Main eigenvectors of the sensitivity matrix|Main eigenvectors of the sensitivity matrix

IASI-NG L2 SO2

Details on 3 10 095 Add the following sequence `3 10 095` : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2) Table references | Element name | note | -- | -- | -- | 3 10 092|IASI NG L2 common information part|The common information shared for CO, NAC, O3 and SO2. 0 08 046|Atmospheric chemical or physical constituent type |8=SO2 0 40 068|General retrieval quality flag for SO2|General retrieval quality flag for SO2 0 10 073|Height of plume|Altitude of the plume 0 15 007|Estimated amount of chemical constituent|SO2 amount at the retrieved plume altitude 0 12 169|Brightness temperature difference|Indicative brightness temperature difference in K 1 02 005|Repeat 2 descriptor 5 times|5 different altitude / pressure 0 07 004|Pressure|Static pressure at different altitude 0 15 007|Estimated amount of chemical constituent|Amount of chemical constituent (DU)

VII L2 common information part

Add the following sequence 3 10 096 : Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2)

Table references Element name note
3 01 005 Originating centre/sub-centre
0 01 007 Satellite identifier
0 02 019 Satellite Instruments
0 02 020 Satellite classification
0 25 060 Software identification
3 01 011 Year, month, day
3 01 013 Hour, minute, second
0 05 040 Orbit number
0 05 001 Latitude (high accuracy)
0 06 001 Longitude (high accuracy)
2 02 126 Change scale
0 07 001 Height of Station
2 02 000 Cancel
0 07 024 Satellite zenith angle
0 05 021 Bearing or azimuth (degree true)
0 07 025 Solar zenith angle
0 05 022 Solar azimuth (degree true)
0 33 028 Snapshot overall quality Processing condition indicator (0=nominal, 1=General degradation)

VII L2 CLD

Details on 3 10 097 Add the following sequence `3 10 097` : Metop-SG Visible Infrared Imager (VII), level 2 cloud cloud detection (CLD) Table references | Element name | note | -- | -- | -- | 3 10 096|VII L2 common information part|The common information shared for CLD, CTP, OCA, WVV and WVI 0 08 011|Meteorological feature|Set to 11 to specify we are speaking about cloud clear 0 33 007|Per cent confidence|Pixel clear-sky confidence 0 08 011|Meteorological feature|Set to cancel 0 08 072|Pixel type|Cloudy flag 0 08 029|Pixel surface type|Surface type

VII L2 CTP

Details on 3 10 098 Add the following sequence `3 10 098` : Metop-SG Visible Infrared Imager (VII), level 2 Cloud Top Pressure (CTP) Table references | Element name | note | -- | -- | -- | 3 10 096|VII L2 common information part|The common information shared for CLD, CTP, OCA, WVV and WVI 0 13 094|Log 10 of Cloud optical thickness|Cloud optical thickness in Log10 0 20 016|Pressure at top of cloud|Pressure at cloud top 0 20 014|Height of top of cloud|Height at cloud top 0 08 093|Measurement uncertainty significance |Total uncertainty 0 13 094|Log 10 of Cloud optical thickness|Error on Cloud optical thickness in Log10 0 20 016|Pressure at top of cloud|Error on pressure at cloud top 0 20 014|Height of top of cloud|Error on Height at cloud top 0 08 093|Measurement uncertainty significance |Cancel total uncertainty

VII L2 OCA

Details on 3 10 099 Add the following sequence `3 10 099` : Metop-SG Visible Infrared Imager (VII), level 2 Optimal Cloud Analysis (OCA) Table references | Element name | note | -- | -- | -- | 3 10 096|VII L2 common information part|The common information shared for CLD, CTP, OCA, WVV and WVI 0 25 144|Number of visual channels used in processing|From 0 to 11 0 25 145|Number of infrared channels used in processing|From 5 to 9 0 20 139|Final cloud model assumption|Final cloud model assumption: 0=clear ,1=single layer water,2=single layer ice,3=2-layers (water/ice),4=single layer ash,5=2-layers ash 0 08 029|Pixel surface type|Surface type 1 20 002|Repeat 20 descriptors 2 times|Loop other first layer or/and second layer 0 08 002|Vertical significance (surface observations)|On first iteration : 21 First instrument detected cloud layer, for second iteration : 22 Second instrument detected cloud layer 0 13 094|Log 10 of Cloud optical thickness|Cloud optical thickness of single or top layer in Log10 0 20 016|Pressure at top of cloud|Pressure at cloud top 0 12 001|Temperature|Temperature at cloud top 2 02 131|Add 3 to scale| 2 01 139|Add 11 to width| 0 20 131|Effective radius of cloud hydrometeors|Cloud particle effective radius 2 01 000|Cancel| 2 02 000|Cancel| 0 20 014|Height of top of cloud|Height at cloud top 0 08 093|Measurement uncertainty significance |Total uncertainty 0 13 094|Log 10 of Cloud optical thickness|Error on cloud optical thickness in log 10 0 20 016|Pressure at top of cloud|Error on Pressure at top cloud 2 02 131|Add 3 to scale| 2 01 139|Add 11 to width| 0 20 131|Effective radius of cloud hydrometeors|Error on cloud particle effective radius 2 01 000|Cancel| 2 02 000|Cancel| 0 20 014|Height of top of cloud|Error on height at cloud top 0 08 093|Measurement uncertainty significance |Cancel total uncertainty

VII L2 WVI/WVV

Details on 3 10 100 Add the following sequence `3 10 100` : Metop-SG Visible Infrared Imager (VII), level 2 Total precipitable water IR and VIS (WVI/WVV) Table references | Element name | note | -- | -- | -- | 3 10 096|VII L2 common information part|The common information shared for CLD, CTP, OCA, WVV and WVI 2 02 129|Add 1 to scale| 2 01 132|Add 4 to width| 0 13 060|Total accumulated precipitation| 2 01 000|Cancel| 2 02 000|Cancel| 0 33 118|Relative error percentage|Relative error on total accumulated precipitation water 0 33 119|Forecast error percentage|Relative difference between retrieved TPW and forecast TPW

Acronyms :

ADC | Analog-to-digital converter ASCAT | Advanced Scatterometer (190 in Common Code Table C-8) DEM | Digital elevation model EOP | Earth orientation parameters EPS-SG | EUMETSAT Polar System – Second Generation EUMETSAT | European Organisation for the Exploitation of Meteorological Satellites FOVs | Fields of view IASI-NG | Infrared Atmospheric Sounding Interferometer - New Generation (231 in Common Code Table C-8) ICI | Ice Cloud Imager (236 in Common Code Table C-8) IERS | International Earth Rotation Service L0 | Level 0 (raw data) L2 | Level 2 (geophysical data) LOS | Line of sight LSM | Landform segmentation model MR | Main reflector MWI | Microwave Imager (237 in Common Code Table C-8) MWS | Microwave Sounder (233 in Common Code Table C-8) NAVATT | Navigation and attitude data NEDT | Nose equivalent delta temperature NEdT | Nose equivalent delta temperature NEΔT | Nose equivalent delta temperature OBCT | On board calibration target PCS | Principle component scores PRT | Platinum resistance thermometer RAD | Radiances RFI | Radio frequency interference RMS | Root mean square SCA | Scatterometer (238 in Common Code Table C-8) SVR | Resampled at 12.5 km swath grid SZF | Resampled at full resolution THM | Thermistor TM | Telemetry TWV | IASI-NG temperature and humidity profiles YPR | Yaw, pitch, and roll YSM | Yaw steering mode

Comments

No response

Requestor(s)

EUMETSAT :

Stakeholder(s)

ECMWF WMO ECCODES

Publication(s)

New entries in : Table B New entries in : Table D New entries in : Flag Table New entries to existing flag table

Expected impact of change

MEDIUM

Collaborators

EUMETSAT scientists

References

No response

Validation

No response

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference.25.26.April.2023 notes:

[ACTION] Anna coordinate a side meeting to discuss further details with Antoine, Simon, Marijana, Sibylle, Sergio and other invitees as desired after Antoine indicates that it's available.

Edit : Antoine fixed a typo in the list displayed in markdown langage. Edit 2 : Antoine added so miss reco on CLD and TWV

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference.2&3May.2023 notes: Antoine presented GHG and CO, but work is still in progress on feedback received during last meeting. @amilan17 will try to set up mtg for 15 May week.

antoinemerle commented 1 year ago

TWV review

(Marijana) asked for clarifications on several elements; The "in bounds check" might fit somewhere else, e.g. significance table [..?..] [ACTION] Antoine double check how many fields for "Satellite instruments" are needed and add notes;

(Sergio) questions about pressures and the relationship to air temperature


CLD review

(Antoine) everything is the same up to as TWV until atmospheric profile (Marjana) Add a significance on the replications of the 2 n_cld to explicit explain what those 2 level correspond too , Antoine will get close to the EUM scientist to find that info and if needed will create a new code table for that one

SFC overview

(Sibylle) Pressure should be from category 10, not cat 07, because this is the measurement of the pressure. Cat 07 is to indicate pressure level at a location and here it's not the location, but the measurement;

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/16.May.2023.satellite.issues/ notes: Antoine provided an overview of recent changes based on feedback and described in comment above.

@antoinemerle actions:

antoinemerle commented 1 year ago

Following the meeting of 16/05/2023 :

Hi @marijanacrepulja and @amilan17 : I updated the proposition according to the presentation I made yesterday. You can now see in the current status the correction applied.


GHG

📝 summary

(Marjana) : for GHG Replicate 1 descriptors 5 times , how the users can know? (Antoine) : corresponding to different layers ( Marjana) :maybe using a pressure descriptors or altitude would be needed (Antoine) : I will investigate on our side and speak to our scientist.

🚀 action :


TWV

📝 : summary

(Sybille) : TWV check the code for first order statistics maybe we can use another one (for TWV, CLD, SFC..etc)

🚀 action :


VII L2 CTP

📝 : summary

(Sybille) : Later in the proposition CTP for VII, title says temperature but there is no temperature.... (Simon) : Yes that's a typo from our side, it is pressure

🚀 action :


marijanacrepulja commented 1 year ago

Hi @antoinemerle and @SimonElliottEUM

I had a chat with scientist and it would be beneficial to have following descriptors in the templates for O3, CO, SO2:

We would appreciate if you can add the pressure [Pa] of the plume alitude in addition to 0 07 XXA 'Height of the plume' for SO2 BUFR template.

Many thanks

antoinemerle commented 1 year ago

Hi @marijanacrepulja,

Just to let you know that I asked those questions to our scientists, and I am waiting their feedback. I am going to update the sequences accordingly to what they can provide to us.

Only one quick question concerning the last bullet point :

Averaging kernels informations, descriptors 015043, 015044 and related pressure 010004 Can you describe what you are excepting there exactly ? I am 100% sure that I understood your request.

Thanks for your feedback and review, @marijanacrepulja !

Best,

marijanacrepulja commented 1 year ago

Hi @antoinemerle and @SimonElliottEUM

Many thanks for feedback.

As for averaging kernels and related pressure, we would need as described below:

015043      numberOfAveragingKernelLayers 102000      102000 Replicate 2 descriptors      031001      delayedDescriptorReplicationFactor   010004      nonCoordinatePressure    015044      averagingKernelValue   

antoinemerle commented 1 year ago

Hi @marijanacrepulja ,

On GHG

on O3, CO and SO2 :

In the case of IASI-NG, unlike IASI, the products are multiple files and not just one, so not all the variables are present in all the products. Unfortunately we don't have what you request in the given product, however it can be find in :

Still under investigation

Only for SO2 :

Still under investigation but I decided to add the pressure because it is likely to be available as static variable.

antoinemerle commented 1 year ago

HI @SibylleK,

Following your comments on : TWV.

Antoine needs to investigate on that possibility to use this 008093 : Measurement uncertainty significance instead of using 0 08 023 : First-order statistics and to add a new element 15 as Algorithm uncertainty

First : THanks a lot for the comment and by investigating I found out it was not for TWV but on VII L2 products. Here is the impacted part of the sequence :

Table references Element name note
0 13 165 log 10 of Cloud optical thickness Cloud optical thickness in Log10
0 20 016 Pressure at top of cloud Pressure at cloud top
0 20 014 Height of top of cloud Temperature at cloud top
0 08 023 First order statistics Specifying we are speaking about : Error / algorithm uncertainty on the following descriptors :
0 13 165 log 10 of Cloud optical thickness Error on Cloud optical thickness in Log10
0 20 016 Pressure at top of cloud Error on Pressure at cloud top
0 20 014 Height of top of cloud Error on Temperature at cloud top
0 08 023 First order statistics Missing

It is a good remark, and I would be happy to avoid creating a new line in this code table if we can use a different descriptor. Basically what the scientists explained to me is : the first iteration is about the geophysical quantity retrieved and then we have its associated algorithm uncertainty also retrieved.

Maybe we can use your descriptors and use the value 2 Random component of uncertainty or maybe another descriptors would be even better ? What do you think ?

Thanks a million for your help @SibylleK

marijanacrepulja commented 1 year ago

Hi @antoinemerle and @SimonElliottEUM

Many thanks for the feedback.

As for GHG

Could you please investigate is it possible to include informations for the a -priori and the average kernels.

  1. We would need for the a -priori Pressure SignificandOfVolumetricMixingRatio

  2. We would need the Error in the integratedMassDensity

  3. We need pressure at the top and at the bottom of the layer for the 0 15 021 | Integrated mass density

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference.6.7.June.2023 notes: Antoine presented the remaining products;

antoinemerle commented 1 year ago

Dear @amilan17 ,

Following the meeting 06/06/2023 : and last comment

thanks,

antoinemerle commented 1 year ago

@amilan17 ,

Dear @amilan17 : bellow and in the initial request I updated the code to be the following :

TABLE B

F X Y Element name Unit Scale Reference value Bit width Instrument
0 12 083 Temperature difference between observed and computed K 2 -10000 16 IASI-NG L2
0 10 073 Height of plume m 2 0 21 IASI-NG L2
0 08 071 Type of cost estimation Code table 0 0 3 IASI-NG L2
0 12 169 Brightness temperature difference K 3 -100000 18 IASI-NG L2
0 15 007 Estimated amount of chemical constituent DU 2 0 16 IASI-NG L2
0 33 116 First guess in bounds check Flag Table 0 0 17 IASI-NG L2
0 33 117 In bounds check Flag Table 0 0 17 IASI-NG L2
0 40 080 Cloud test results for IASI-NG level 2 Flag Table 0 0 5 IASI-NG L2
0 40 075 Retrieval error covariance matrix for humidity in principal component domain Numeric 0 0 21 IASI-NG L2
0 40 076 Retrieval error covariance matrix for temperature in principal component domain Numeric 0 0 21 IASI-NG L2
0 25 144 Number of visual channels used in processing Numeric 0 0 6 VII-02-OCA
0 25 145 Number of infrared channels used in processing Numeric 0 0 6 VII-02-OCA
0 20 139 Final cloud model assumption Code table 0 0 4 VII-02-OCA
0 13 094 log 10 of Cloud optical thickness Numeric 3 -2000 14 VII-02-CTP,VII-02-OCA
0 33 118 Relative error percentage Percentage 2 -10000 15 VII-02-WVV, VII-02-WVI
0 33 119 Forecast error percentage Percentage 0 -100 8 VII-02-WVV, VII-02-WVI

TABLE D

New element Element name
3 10 087 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (TWV, CLD, SFC, GHG)
3 10 088 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 temperature and water vapour profiles (TWV)
3 10 089 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 cloud detection (CLD)
3 10 090 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 surface parameters (SFC)
3 10 091 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 total columns of nitrous oxide, methane and carbon dioxide (GHG)
3 10 092 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 common information part (IASI NG L2 CO_, NAC, O3, SO2)
3 10 093 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 carbon monoxide total column, carbon monoxide profile (CO_)
3 10 094 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 nitric acid total column, nitric acid profile (NAC) / level 2 ozone total column, ozone profile (O3_)
3 10 095 Metop-SG Infrared Atmospheric Sounder Interferometer - New Generation (IASI-NG), level 2 SO2 total column (SO2)
3 10 096 Metop-SG Visible Infrared Imager (VII), level 2 common part for VII L2 CLD, CTP, OCA, WVI, WVV
3 10 097 Metop-SG Visible Infrared Imager (VII), level 2 cloud cloud detection (CLD)
3 10 098 Metop-SG Visible Infrared Imager (VII), level 2 Cloud top pressire (CTP)
3 10 099 Metop-SG Visible Infrared Imager (VII), level 2 Optimal Cloud Analysis (OCA)
3 10 100 Metop-SG Visible Infrared Imager (VII), level 2 Total precipitable water IR and VIS (WVI/WVV)

can you confirm this is okay for you thanks a million !

antoinemerle commented 1 year ago

Hi @amilan17,

I just pushed all the changes on the dedicated branch : 03b301f75241c5411d61ca7788989fd4577b704e

Can you provide me the Slideshare URL in order to allow me to drop the sample files there ?

Thanks a million !

antoinemerle commented 1 year ago

HI @amilan17 ,

Here are all the data generated corresponding to this issue and to the changes applied to this branch. I already put version 40 for the Master table code in the test data

Note : Due to the Github's restrictions of not being able to upload more than 25MB I had to split the VII L2 in 3 parts... sorry for the inconvenience....

IASI 2 :

VII L2 :

marijanacrepulja commented 1 year ago

Hi @antoinemerle,

Many thanks for the samples provided.

I believe the units should be corrected for the descriptors

<!DOCTYPE html>

0 33 118 Relative error percentage Percentage 2 -10000 15 VII-02-WVV, VII-02-WVI
0 33 119 Forecast error percentage Percentage 0 -100 8 VII-02-WVV, VII-02-WVI

Instead of Percentage should be %

antoinemerle commented 1 year ago

Hi, @marijanacrepulja ,

Thanks for spotting it! Updating 033118 and 03119 from percentage to % in de70ce8

SibylleK commented 1 year ago

BUFR-issue161-EPS-SG-Level2_DWDreadbufrxOutput.tar.zip

Using the BUFR tables of the branch (and link version 41 to 40, as the version in the example is 40), the examples could be read with the DWD BUFR reading software. But only the first 100 Subsets of each BUFR are read, to be able to upload the output here. The Output of the BUFR seems to be Ok. (But I don't know whether the values are correct. Please feel free to check:-))

There are no examples with 310090 (IASI SFC?) and 310098 (VII L2 CTP is encoded with 310097 as the CLD) in the provided test-BUFRs. All other sequences are included in the examples and the BUFR are readable.

From my point of view the validation of the sequences 310087, 310088, 310089, 310091, 310092, 310093, 310094, 310095, 310096, 310097, 310099, 310100 including table B entries and code flag table entries is done.

@antoinemerle can you provide samples for 310090 and 310098?

amilan17 commented 1 year ago

https://github.com/wmo-im/CCT/wiki/Teleconference.13.July.2023 notes:

@antoinemerle address comments in PR, provide two more samples for antoine and @SibylleK validate within a week;

antoinemerle commented 1 year ago

Hi @SibylleK ,

Again, apologies for not attending the meeting the other day and for not having replied to you before the meeting ! 😢

In any case, to avoid more troubles and confusion : please find below the two product for your review 😃

Thanks again for your time to decode and review my product @SibylleK ! 👍

SibylleK commented 1 year ago

Thanks @antoinemerle for the new examples with the two remaining sequences.

Here is the output of the DWD software: W_XX-EUMETSAT-Darmstadt,SAT,SGA1-IAS-02-SFC_C_EUMT_20230718092907_G_O_20070912101146_20070912101417_O_N____.bin.DWDreadbufrxOutput.zip W_XX-EUMETSAT-Darmstadt,SAT,SGA1-VII-02-CTP_C_EUMT_20230718110136_G_D_20070912084404_20070912084405_T_B____.bin.DWDreadbufrxOutput100.zip

I only read the first 100 subset of the first BUFR of the provided CTP BUFR examples. But this should be enough for a successful validation.

Therefore, from my point of view, validation is done.

SimonElliottEUM commented 1 year ago

[like] Simon Elliott reacted to your message:


From: Sibylle Krebber @.> Sent: Wednesday, July 19, 2023 3:48:52 PM To: wmo-im/BUFR4 @.> Cc: Simon Elliott @.>; Mention @.> Subject: Re: [wmo-im/BUFR4] Proposal for new BUFR table entries for EPS-SG (EUMETSAT) : IASI-NG Level 2 and VII Level 2 (Issue #161)

WARNING: This email originated outside the organisation. Treat links and attachments with caution.

Thanks @antoinemerlehttps://github.com/antoinemerle for the new examples with the two remaining sequences.

Here is the output of the DWD software: W_XX-EUMETSAT-Darmstadt,SAT,SGA1-IAS-02-SFC_C_EUMT_20230718092907_G_O_20070912101146_20070912101417_O_N.bin.DWDreadbufrxOutput.zip<https://github.com/wmo-im/BUFR4/files/12097114/W_XX-EUMETSAT-Darmstadt.SAT.SGA1-IAS-02-SFC_C_EUMT_20230718092907_G_O_20070912101146_20070912101417_O_N.bin.DWDreadbufrxOutput.zip> W_XX-EUMETSAT-Darmstadt,SAT,SGA1-VII-02-CTP_C_EUMT_20230718110136_G_D_20070912084404_20070912084405_T_B.bin.DWDreadbufrxOutput100.zip<https://github.com/wmo-im/BUFR4/files/12097128/W_XX-EUMETSAT-Darmstadt.SAT.SGA1-VII-02-CTP_C_EUMT_20230718110136_G_D_20070912084404_20070912084405_T_B.bin.DWDreadbufrxOutput100.zip>

I only read the first 100 subset of the first BUFR of the provided CTP BUFR examples. But this should be enough for a successful validation.

Therefore, from my point of view, validation is done.

— Reply to this email directly, view it on GitHubhttps://github.com/wmo-im/BUFR4/issues/161#issuecomment-1642340977, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO4JDOQWU5MK3VWTAFIBIP3XQ76WJANCNFSM6AAAAAAXLQZQWQ. You are receiving this because you were mentioned.Message ID: @.***>

Any email message from EUMETSAT is sent in good faith but shall neither be binding nor construed as constituting a commitment by EUMETSAT, except where provided for in a written agreement or contract or if explicitly stated in the email. Please note that any views or opinions presented in this email are solely those of the sender and do not necessarily represent those of EUMETSAT. This message and any attachments are intended for the sole use of the addressee(s) and may contain confidential and privileged information. Any unauthorised use, disclosure, dissemination or distribution (in whole or in part) of its contents is not permitted. If you received this message in error, please notify the sender and delete it from your system.

amilan17 commented 1 year ago

@antoinemerle  are IASI-NG L2 GHG and IASI-NG L2 SO2 still under investigation for pressure?

antoinemerle commented 1 year ago

HI @amilan17 ,

At the moment, this information is still pending on our side. The scientists were not able to give a clear answer to that question and are still investigating.

The way I propose is to let the comment related to pressure for SO2 and GHG empty at the moment and when we will have this info, I will submit request to add the correct note in the corresponding "comment" column.

What do you think @amilan17 ?

Thanks !

Best,

antoinemerle commented 1 year ago

Hi @SibylleK ,

Thank you for your work and dedication on that Task ! Much appreciated !

Cheers,

Thanks @antoinemerle for the new examples with the two remaining sequences.

Here is the output of the DWD software: W_XX-EUMETSAT-Darmstadt,SAT,SGA1-IAS-02-SFC_C_EUMT_20230718092907_G_O_20070912101146_20070912101417_O_N____.bin.DWDreadbufrxOutput.zip W_XX-EUMETSAT-Darmstadt,SAT,SGA1-VII-02-CTP_C_EUMT_20230718110136_G_D_20070912084404_20070912084405_T_B____.bin.DWDreadbufrxOutput100.zip

I only read the first 100 subset of the first BUFR of the provided CTP BUFR examples. But this should be enough for a successful validation.

Therefore, from my point of view, validation is done.