search

GAD-DIMNT-CPTEC / readDiag

readDiag is a Python package that provides a class to read and plot the Gridpoint Statistical Interpolation (GSI) diagnostics files. It can be used to retrieve and investigate important information from the data assimilation process
https://gad-dimnt-cptec.github.io/readDiag
Other
0 stars 3 forks source link

Corrigindo e atualizando o dicionário dos dados usados na assimilação #41

Closed sapucci closed 3 months ago

sapucci commented 5 months ago

Corrigir o dicionário dos tipos de dados, com bug nos nos nomes das fontes de dados, e atualizar a base de dados para acrescentar os dados adicionais entre as versões 3.3 e 3.7 do GSI. Nessa versão também será considerada tanto a table do trunk como uma tabela como o arquivo table.yml criado como uma solução do problema reportado na issue #28.

Enfase é dados para os dados de radio ocultação GNSS e dados de radiância, o ultimo apenas na table.yml

Algumas atividades nessa linha foram reportadas na wiki do SMNA no redMine, mas serão reescrita aqui para conforto dos usuarios e registro das atividades.

sapucci commented 3 months ago

Inicialmente foi verificado no prepbufr o que é esse tipo de dado.

NO NCEP da descrição do prebur no site https://www.nco.ncep.noaa.gov/sib/decoders/BUFRLIB/toc/dfbfmstab/#cftables

No link https://www.nco.ncep.noaa.gov/sib/decoders/BUFRLIB/toc/dfbfmstab/bufrtab.CodeFlag_LOC_0_7_1

tem a lista dos Mimimonicos usados no prepburs que á lista que segue.

0-55-007 | TYP ; CODE | 102 > | MASS Report - SSM/I 7-channel brightness temperatures | 111 > | MASS Report - Synthetic (bogus) tropical cyclone storm center | 112 > | MASS Report - Pseudo mean sea-level pressure at tropical cyclone storm center | 120 > | MASS Report - Rawinsonde | 122 > | MASS Report - Class sounding | 126 > | MASS Report - RASS temperature profile | 130 > | MASS Report - AIREP or PIREP aircraft | 131 > | MASS Report - ASDAR aircraft | 132 > | MASS Report - Flight-level reconnaissance and profile dropsonde | 133 > | MASS Report - MDCRS ACARS aircraft | 134 > | MASS Report - TAMDAR aircraft | 135 > | MASS Report - Canadian AMDAR aircraft | 150 > | MASS Report - SSM/I superobed (1 degree lat/long) FNMOC rain rate | 151 > | MASS Report - NESDIS 1x1 cloud top pressure, temperature and cloud amount | 152 > | MASS Report - SSM/I superobed (1 degree lat/long) Neural net-3 precipitable water over ocean | 153 > | MASS Report - GPS integrated precipitable water | 156 > | MASS Report - NESDIS 1x1 field-of-view 4-layer precipitable water over land - clear | 157 > | MASS Report - NESDIS 1x1 field-of-view 4-layer precipitable water over land - cloudy | 158 > | MASS Report - NESDIS 1x1 field-of-view 4-layer precipitable water over ocean - clear | 159 > | MASS Report - NESDIS 1x1 field-of-view 4-layer precipitable water over ocean - cloudy | 161 > | MASS Report - NESDIS ATOVS virtual temperature soundings at 250 km resolution over land - clear | 163 > | MASS Report - NESDIS ATOVS virtual temperature soundings at 250 km resolution over land - cloudy | 164 > | MASS Report - NESDIS 1x1 field-of-view radiances water over land - clear | 165 > | MASS Report - NESDIS 1x1 field-of-view radiances water over land - cloudy | 171 > | MASS Report - NESDIS ATOVS virtual temperature soundings at 250 km resolution over ocean - clear | 173 > | MASS Report - NESDIS ATOVS virtual temperature soundings at 250 km resolution over ocean - cloudy | 174 > | MASS Report - NESDIS 1x1 field-of-view radiances water over ocean - clear | 175 > | MASS Report - NESDIS 1x1 field-of-view radiances water over ocean - cloudy | 180 > | MASS Report - Surface marine with reported station pressure (SHIP, BUOY, C-MAN or tide gauge) | 181 > | MASS Report - Surface land with reported station pressure (SYNOP, METAR) | 182 > | MASS Report - Splash-level dropsonde over ocean | 183 > | MASS Report - Surface marine with missing station pressure (SHIP, BUOY, C-MAN or tide gauge) | 187 > | MASS Report - Surface land with missing station pressure (SYNOP, METAR) | 188 > | MASS Report - Surface mesonet | 190 > | MASS Report - OPC/NOS point mean sea-level pressure bogus over ocean | 191 > | MASS Report - Australian PAOB mean sea-level pressure bogus over ocean | 210 > | WIND Report - Synthetic (bogus) tropical cyclone | 220 > | WIND Report - Rawinsonde | 221 > | WIND Report - PIBAL | 222 > | WIND Report - Class sounding | 223 > | WIND Report - NOAA Profiler Network (NPN) wind profiler | 224 > | WIND Report - NeXRaD Vertical Azimuth Display (VAD) | 227 > | WIND Report - Multi-agency (MAP) wind profiler | 228 > | WIND Report - Japanese Meteorological Agency (JMA) wind profiler | 229 > | WIND Report - Wind profiler from PILOT | 230 > | WIND Report - AIREP or PIREP aircraft | 231 > | WIND Report - ASDAR aircraft | 232 > | WIND Report - Flight-level reconnaissance and profile dropsonde | 233 > | WIND Report - MDCRS ACARS aircraft | 234 > | WIND Report - TAMDAR aircraft | 235 > | WIND Report - Canadian AMDAR aircraft | 241 > | WIND Report - India infrared and visible cloud drift | 242 > | WIND Report - Japan infrared and visible cloud drift at levels below 850 mb | 243 > | WIND Report - EUMETSAT infrared and visible cloud drift at levels below 850 mb | 245 > | WIND Report - NESDIS infrared cloud drift | 246 > | WIND Report - NESDIS imager water vapor - cloud top | 247 > | WIND Report - NESDIS imager water vapor - deep layer | 248 > | WIND Report - NESDIS sounder water vapor - cloud top | 249 > | WIND Report - NESDIS sounder water vapor - deep layer | 250 > | WIND Report - Japan imager water vapor, cloud top and deep layer | 251 > | WIND Report - NESDIS visible cloud drift | 252 > | WIND Report - Japan infrared and visible cloud drift at levels above 850 mb | 253 > | WIND Report - EUMETSAT infrared and visible cloud drift at levels above 850 mb | 254 > | WIND Report - EUMETSAT imager water vapor, cloud top and deep layer | 255 > | WIND Report - NESDIS picture triplet cloud drift | 256 > | WIND Report - India imager water vapor | 257 > | WIND Report - MODIS/POES infrared cloud drift | 258 > | WIND Report - MODIS/POES imager water vapor - cloud top | 259 > | WIND Report - MODIS/POES imager water vapor - deep layer | 280 > | WIND Report - Surface marine with reported station pressure (SHIP, BUOY, C-MAN or tide gauge) | 281 > | WIND Report - Surface land with reported station pressure (SYNOP, METAR) | 282 > | WIND Report - ATLAS BUOY | 283 > | WIND Report - SSM/I superobed (1 degree lat/long) Neural net-3 wind speed over ocean | 284 > | WIND Report - Surface marine (SHIP, BUOY, C-MAN or tide gauge) or surface land (SYNOP, METAR) with missing station pressure | 285 > | WIND Report - Superobed (0.5 degree lat/long) scatterometer winds over ocean (QUIKSCAT) | 286 > | WIND Report - Scatterometer winds over ocean (ERS) | 287 > | WIND Report - Surface land (METAR) with missing station pressure | 288 > | WIND Report - Surface mesonet | 289 > | WIND Report - Superobed (1 degree lat/long) scatterometer winds over ocean (WINDSAT) | 290 | WIND Report - Non-superobed scatterometer winds over ocean (ASCAT)

Nela consta o 255 como sendo | 255 > | WIND Report - NESDIS picture triplet cloud drift

sapucci commented 3 months ago

Uma lista de outros tipos de dados kx não são encontrados de precisa ser ajustado.

na pressão o Kx 132 não é encontrado

no sst o kx 187 não é encontrado.

no GPS a lista fica muito maior

a lista é 269 751 755 753 750 66 825 44 do que foi listado na rodada das /share/das/dist/luiz.sapucci/SMNA/EXP20/2024021900/diag_conv_01.2024021900 apenas o kx 3 e 42 já estava na lista que era o metop-B e o terrasar respectivamente.

É preciso atualizar a table do diag. Isso foi feito para o kx 255 par ao uv e a figura abaixo foi gerado corretamente.

ro_gsi_3 7

sapucci commented 3 months ago

Para os dados de radiocultação a lista de dados do GSI na versão anterior era

! dtype type sub iuse ... READ_CONVINFO: gps 4 0 1 METOP-A READ_CONVINFO: gps 41 0 -1 CHAMP READ_CONVINFO: gps 722 0 1 GRACE-A READ_CONVINFO: gps 723 0 -1 GRACE-B READ_CONVINFO: gps 740 0 1 COSMIC (FM1) READ_CONVINFO: gps 741 0 1 COSMIC (FM2) READ_CONVINFO: gps 742 0 1 COSMIC (FM3) READ_CONVINFO: gps 743 0 1 COSMIC (FM4) READ_CONVINFO: gps 744 0 1 COSMIC (FM5) READ_CONVINFO: gps 745 0 1 COSMIC (FM6) READ_CONVINFO: gps 820 0 1 SAC-C READ_CONVINFO: gps 42 0 1 TerraSAR-X READ_CONVINFO: gps 43 0 -1 Tandem-X READ_CONVINFO: gps 786 0 1 C/NOFS READ_CONVINFO: gps 421 0 -1 OCEANSAT-2 READ_CONVINFO: gps 3 0 1 METOP-B READ_CONVINFO: gps 821 0 -1 SAC-D/Aquarius
READ_CONVINFO: gps 440 0 -1 Megha-Tropiques

como descrito pela Ivette na wiki

https://projetos.cptec.inpe.br/projects/smg/wiki/Rognss

sapucci commented 3 months ago

Olhando para a lista da configuração do EXP20 temos no arquivo convinfo usado

Não usado (monitorados):

!otype type sub iuse twindow numgrp ngroup nmiter gross ermax ermin var_b var_pg ithin rmesh pmesh npred pmot ptime ib ip gps 041 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 ChamP gps 066 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 Indefinido gps 266 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 Indefinido gps 267 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 Indefinido gps 268 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 Indefinido gps 440 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 MegaTropic gps 421 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 OCEAN-SAT2 gps 724 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2A sat.P1 gps 725 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2A sat.P2 gps 726 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2A sat.P3 gps 727 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2A sat.P4 gps 728 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2A sat.P5 gps 729 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2A sat.P6 gps 821 0 -1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 SAC-D

Dados usados pelo global.convinfo do GSI 3.7 !otype type sub iuse twindow numgrp ngroup nmiter gross ermax ermin var_b var_pg ithin rmesh pmesh npred pmot ptime ib ip gps 004 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 MetOp-B gps 722 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 GRACE-A gps 723 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 GRACE-B gps 740 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-1 sat.P1 gps 741 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-1 sat.P2 gps 742 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-1 sat.P3 gps 743 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-1 sat.P4 gps 744 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-1 sat.P5 gps 745 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-1 sat.P6 gps 820 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 SAC-C gps 042 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 Terra-SAR-X gps 043 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 TanDEN-X gps 786 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 C/NOFS gps 003 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 MetOp-B gps 825 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 KOMPSAT-5
gps 750 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2B sat.E1 gps 751 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2B sat.E2 gps 752 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2B sat.E3 gps 753 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2B sat.E4 gps 754 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2B sat.E5 gps 755 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 COSMIC-2B sat.E6 gps 044 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 PAZ gps 005 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 MetOp-C gps 265 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 Indefinido gps 269 0 1 3.0 0 0 0 10.0 10.0 1.0 10.0 0.000000 0 0. 0. 0 0. 0. 0 0 Indefinido

A lista dos satélites colocados aqui como indefinido é por que não constam na table18 do NCEP. Uma nova fonte mais atualizada vai ser preciso para identificar esses minimonicos.

sapucci commented 3 months ago

Uma lista de minimonuos do NCEP para os prepbufr foi encontrados no site

https://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/document.htm

https://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/document.htm#prepdata

Onde a relação dos dados de rO GSP estão em na tabela 18 no site

https://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_18.htm

Fernando Sapucci, Luiz escreveu:

Uma lista de minimonuos do NCEP para os prepbufr foi encontrados no site

https://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/document.htm

https://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/document.htm#prepdata

Onde a relação dos dados de rO GSP estão em na tabela 18 no site

https://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_18.htm

Baseado nessa tabela o nos dados usados no global.convinfo foi atualizado a table do readDiag. no que se refere aos dados GPS de radio ocultation tem-se:

3; gps; GPS; Sensor RO GRASS in the MetOp-B satellite; #297df1; 8; 1 4; gps; GPS; Sensor RO GRASS in the MetOp-A satellite; #fe6f22; s; 1 5; gps; GPS; Sensor RO GRASS in the MetOp-C satellite; #fe6f22; x; 1 41; gps; GPS; RO data from CHAMP satellite; #fe6f22; x; -1 42; gps; GPS; RO data from TerraSAR-X satellite; #fe6f22; x; 1 43; gps; GPS; RO data from TanDEM-X satellite; #fe6f22; x; 1 44; gps; GPS; RO Data from PAZ satellite; #575738; s; 1 66; gps; GPS; GPS-INFO; #2e8e24; x; -1 265; gps; GPS; GPS-INFO; #2e8e24; x; 1 266; gps; GPS; GPS-INFO; #2e8e24; x; -1 267; gps; GPS; GPS-INFO; #2e8e24; x; -1 268; gps; GPS; GPS-INFO; #2e8e24; x; -1 269; gps; GPS; GPS-INFO; #2e8e24; x; 1 440; gps; GPS; RO Data from Megha-Tropiques satellite; #fe6f22; x; -1 421; gps; GPS; RO data from Oceansat-2 satellite; #fe6f22; x; -1 722; gps; GPS; Sensor RO Data from GRACE-A satellite; #fe6f22; x; 1 723; gps; GPS; Sensor RO data from GRACE-B satellite; #fe6f22; x; 1 724; gps; GPS; Constelation COSMIC-2A Satelitte: P1; #d6616b; 8; -1 725; gps; GPS; Constelation COSMIC-2A Satelitte: P2; #de57fb; o; -1 726; gps; GPS; Constelation COSMIC-2A Satelitte: P3; #f1d34d; p; -1 727; gps; GPS; Constelation COSMIC-2A Satelitte: P4; #d61d14; P; -1 728; gps; GPS; Constelation COSMIC-2A Satelitte: P5; #2e8e24; x; -1 729; gps; GPS; Constelation COSMIC-2A Satelitte: P6; #e7969c; s; -1 740; gps; GPS; Constelation COSMIC-1 Satelitte: FM1; #42b7a3; ; 1 741; gps; GPS; Constelation COSMIC-1 Satelitte: FM2; #de57fb; o; 1 742; gps; GPS; Constelation COSMIC-1 Satelitte: FM3; #f1d34d; p; 1 743; gps; GPS; Constelation COSMIC-1 Satelitte: FM4; #d61d14; P; 1 744; gps; GPS; Constelation COSMIC-1 Satelitte: FM5; #2e8e24; x; 1 745; gps; GPS; Constelation COSMIC-1 Satelitte: FM6; #2e8e24; x; 1 750; gps; GPS; Constelation COSMIC-2B Satelitte: E1; #42b7a3; ; 1 751; gps; GPS; Constelation COSMIC-2B Satelitte: E2; #de57fb; o; 1 752; gps; GPS; Constelation COSMIC-2B Satelitte: E3; #f1d34d; p; 1 753; gps; GPS; Constelation COSMIC-2B Satelitte: E4; #d61d14; s; 1 754; gps; GPS; Constelation COSMIC-2B Satelitte: E4; #2e8e24; x; 1 755; gps; GPS; Constelation COSMIC-2B Satelitte: E5; #e7969c; s; 1 786; gps; GPS; RO data from C/NOFS satellite; #875BAE; s; 1 820; gps; GPS; RO data from SAC-C satellite; #297df1; 8; 1 821; gps; GPS; RO Data from SAC-D satellite; #297df1; 8; -1 825; gps; GPS; RO data from KOMPSAT-5 satellite; #297df1; 8; 1

sapucci commented 3 months ago

Com essas informações feitas na versão gsidiag/table foi adaptada para a versão gsidiag/table.yml no branch da caroline sendo ajustado em conjunto com a versão nova do readDiag para a radiância. Isso foi feito pois também houveram ajustes nos dados de radiância nessa tabela para o AMSU-A e os ajustes para os dados não existentes da nova versão do GSI3.7 foram incorporadas.

A revisão com ajustes feitos nessa essue estão na revisão https://github.com/GAD-DIMNT-CPTEC/readDiag/commit/3a7b50cbf41ad3c4b0229660df8bf7f6a3990d02

Testes foram feitos e os resultados adequados foram encontrados, logo a tarefa é fechada nesse comentário.