Code table 042017 new quality flags for wave spectra

[ghajduch]

Initial request

With reference to https://confluence.ecmwf.int/display/ECC/WMO%3D27+element+table#WMO=27elementtable-CL_33, the
WMO Master table number 25 defines an element called
ConfidenceOfInversionForEachPartitionOfSwellWaveSpectra in class 42.

This table allows to annotate confidence in the direction of the inversion of swell spectra.
The corresponding code-flag table is available here
https://confluence.ecmwf.int/display/ECC/WMO%3D27+code-flag+table#WMO=27codeflagtableCF_042004

This include information on "Wave direction resolved", "180-degree ambiguity not resolved" and "missing value".

We propose to add new values corresponding to overall quality flags on the swell inversion process for the overall spectra and per partition stating if the inversion is of one of the following qualities: very good, good, medium, low, poor, missing.
The already existing fields cannot be reused as having another meaning.

We checked that the WMO master table number 38 does not contain neither an appropriate element.
We propose two new elements called respectively QualityFlagForEachPartitionOfSwellWaveSpectra and
QualityFlagOfSwellWaveSpectra.
The QualityFlagForEachPartitionOfSwellWaveSpectra aims at annotating the quality of the inversion of swell
spectra for each partition.
The QualityFlagOfSwellWaveSpectra aims at annotating the quality of the inversion of the swell spectra for
the complete spectrum.
Each of those new elements should have values defined in a new code-table defining the following values:

• Very good
• Good
• Medium
• Low
• Poor
• Missing
  As the allowed values for those two elements are the same, we propose that the two elements will be
  defined with reference to the same common code table

Amendment details

Code table 42017 Quality flag of swell wave spectra

code figure	name
0	Good
1	Medium
2	Low
3	Poor
15	Missing value

42018 Quality flag for each partition of swell wave spectra

code figure	name
0	Very good
1	Good
2	Medium
3	Low
4	Poor
15	Missing value

Comments

No response

Requestor(s)

Guillaume Hajduch (Collecte Localisation Satellites), as Service Manager of the SAR Mission Performance Cluster (SAR-MPC) Service, on behalf of European Space Agency (ESA)
Muriel Pinheiro as Technical Officer of the SAR-MPC, and responsible of the QC and Calibration/Validation of Sentinel-1 mission.
Antonio Valentino , as support to Muriel Pinheiro

Stakeholder(s)

The weather forecast community in charge of assimilation of swell observation in swell .
This is inter alia

Meteo France, Lotfi Aouf,
ECMWF Stephen English
ECMWF Carlo Buontempo
ECMWF Jean Bidlot
ECMWF Sean Healy
ECMWF Saleh Abdalla

Publication(s)

Manual on Codes (WMO-No. 306), Volume I.2 BUFR code table 0 42 004

Expected impact of change

MEDIUM

Collaborators

Manuel Goacolou, Collecte Localisation Satellite
Lotfi Aouf, Meteo France

References

No response

Validation

No response

[amilan17]

https://github.com/wmo-im/CCT/wiki/Teleconference.6.7.June.2023 notes: Team has questions about two tables with the same entries....

[ghajduch]

I don't understand the question. What same entries ? There is ConfidenceOfInversionForEachPartitionOfSwellWaveSpectra which in fact contain elements on wave propagation direction (at 180°) being resolved or not. We propose to add QualityFlagForEachPartitionOfSwellWaveSpectra which aims to contain the overall quality of estimation on the swell spectra.

This refer to two different things that cannot be grouped into a single table. Except maybe if the table is a bit field. But for now the ConfidenceOfInversionForEachPartitionOfSwellWaveSpectra is not designed to be a bit field, and changing this may be an issue for users.

[ghajduch]

Dear @amilan17 ,

Can you clarify the "questions about tables with same entry" ? We can provide clarification if required.

[marijanacrepulja]

@ghajduch

Can you confirm that you are proposing to update existing 042004 with the following entries

Code/Flag Figure	Description
3	Very good
4	Good
5	Medium
6	Low
7	Poor

[amilan17]

https://github.com/wmo-im/CCT/wiki/Teleconference.13.July.2023 notes:

waiting for feedback; if the proposal above is acceptable, we can include in FT2023-2, otherwise it will be worked on for FT2024-1

[ghajduch]

Dear @amilan17 , We are not proposing to update the existing 042004, but to add a new line in table of class 42. The information stored in 042004 cannot be mixed with the one we want to add.

To be more explicit, in the class 42, we propose to add the following: FXXYYY = 042017 (or other number, up to you) class = 42 Code = 17 (or other number, up to you) Name = Quality Flag Of Swell Wave Spectra key = QualityFlagOfSwellWaveSpectra Unit = Code table to be added (see below) Scale = 0 Reference = 0 Width = 4 Crex Unit = Code table (see below) Crex scale = 0 Crex width = 2

Code table to be used

Code/Flag Figure	Description
0	Very good
1	Good
2	Medium
3	Low
4	Poor
5-14	Reserved
15	missing

Thanks

[amilan17]

@ghajduch Thank you for the clarification. @marijanacrepulja Do you think this can be included in FT2023-2? If so, can you review the PR?

[marijanacrepulja]

@amilan17 It is definition of a new descriptor. I believe it is good to have a discussion with the team.

[amilan17]

@ghajduch The team still needs time to discuss this proposal when we meet again in September.

[ghajduch]

Dear @amilan17, When do the team meet on this topic in Septembre ? Do you need some further clarifications ? Is it possible to attend to this meeting for this specific topic ?

Thanks

[amilan17]

https://github.com/wmo-im/CCT/wiki/20.to.22.September.2023 notes:

• Guillaume presented the issue; team agrees to the proposal
• Need sample data for validation, Guillaume can provide soon, noting that anytime between now and end of November is ok, but earlier is better

[ghajduch]

Dear @amilan17 ,

Here are the samples for validation:

The sample BUFR S1B_WV_BFR__2SSV_20211213T163303_20211213T165449_030009_039532_4EB4.zip

The original NetCDF files (first 44 of them) source_nc.zip

The updates are:

1/ add to element.table 042017|qualityFlagOfSwellWaveSpectra|table|Quality Flag Of Swell Wave Spectra|CODE TABLE|0|0|4|CODE TABLE|0|2 042018|qualityFlagForEachPartitionOfSwellWaveSpectra|table|Quality Flag For Each Partition Of Swell Wave Spectra|CODE TABLE|0|0|4|CODE TABLE|0|4

Create codetables/42017.table 0 0 GOOD 1 1 LOW 2 2 MEDIUM 3 3 POOR 15 15 MISSING VALUE

create codetables/42018.table 0 0 VERY_GOOD 1 1 GOOD 2 2 LOW 3 3 MEDIUM 4 4 POOR 15 15 MISSING VALUE

[sergioh-pessoal]

I have successfully decoded one of the provided samples. I have not compared the values with the netcdf version yet. I will do this later for all files

Below is the result of the first sample s1b-wv1-mbu-vv-20211213t163332-20211213t163335-030009-039532-003_4EB4.decoded_inpe.txt.gz

[amilan17]

https://github.com/wmo-im/CCT/wiki/Teleconference.21.22.November.2023 notes: Sergio validated; Sergio suggested that two tables with the same values may not be necessary; DECISION - both tables @sergioh-pessoal will update branch: https://github.com/wmo-im/BUFR4/tree/166-code-table-042017-new-quality-flags-for-wave-spectra

[ghajduch]

To avoid any misunderstanding: We (@mgoacolou and I) proposed one table for quality flag on the complete spectra and one table for quality flag per partition. Our understanding of BUFR format specification is that there is no way to share the definition of those quality flags for tables having different names. We do need different names (one for full spectra and one per partition). But maybe we are wrong on this and it is possible to ensure this. If so, let us know. thanks

[sergioh-pessoal]

@ghajduch I got your point . You are right. In the case of this sequence, two tables is the best option.

I will update the descriptor and code table in git branch.

I will use your last update (from 3 weeks ago) According to this last update, the code figure for "VERY GOOD does not exist in table 42017 but it does exist in 42018. Is that right?

[ghajduch]

@sergioh-pessoal , We confirm that "VERY GOOD" shall not exist in 42017, while it exist in 42018.

However, having further look to this, it appears we made a mistake in the definition of the values vs code table. The order should be from high quality to low quality : (Very Good), Good, Medium, Low, Poor

Then

Create codetables/42017.table 0 0 GOOD 1 1 MEDIUM 2 2 LOW 3 3 POOR 15 15 MISSING VALUE

create codetables/42018.table 0 0 VERY_GOOD 1 1 GOOD 2 2 MEDIUM 3 3 LOW 4 4 POOR 15 15 MISSING VALUE

[amilan17]

branch is valid

[sergioh-pessoal]

Just for information, I also checked the values of 42017 and 42018 in the BUFR sample with the corresponding value in the source nc file and they are ok!

[amilan17]

https://github.com/wmo-im/CCT/wiki/Teleconference.21.29.November.2023 notes:

validated