search

NOAA-EMC / GDASApp

Global Data Assimilation System Application
GNU Lesser General Public License v2.1
14 stars 28 forks source link

GHRSST converter is not working properly #1019

Closed guillaumevernieres closed 3 months ago

guillaumevernieres commented 3 months ago

I suspect the datetime is wrong. There's an example of processed obs here:

[hfe03 /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/runs/cp1]$ du -sh cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst*
12M cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst_abi_g16_l3c.2021070200.nc4
9.7M    cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst_abi_g17_l3c.2021070200.nc4
12M cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst_ahi_h08_l3c.2021070200.nc4
1.5M    cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst_avhrr_ma_l3u.2021070200.nc4
1.8M    cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst_avhrr_mb_l3u.2021070200.nc4
2.3M    cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst_viirs_n20_l3u.2021070200.nc4
2.3M    cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/gdas.t00z.sst_viirs_npp_l3u.2021070200.nc4

The datetime doesn't seem correct. The output file after DA are here:

[hfe03 /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/runs/cp1]$ du -sh /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/*
39M /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/adt_rads_all.2021070200.nc4
24M /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/icec_amsr2_north.2021070200.nc4
48M /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/icec_amsr2_south.2021070200.nc4
2.3M    /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/sst_abi_g16_l3c.2021070200.nc4
16K /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/sst_abi_g17_l3c.2021070200.nc4
16K /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/sst_ahi_h08_l3c.2021070200.nc4
60K /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/sst_avhrr_ma_l3u.2021070200.nc4
20K /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/sst_avhrr_mb_l3u.2021070200.nc4
20K /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/sst_viirs_n20_l3u.2021070200.nc4
64K /scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/RUNDIRS/cp1.e/gdasocnanal_00/diags/sst_viirs_npp_l3u.2021070200.nc4

The only non-empty output is ABI on GOES16.

Could you have a look at the datetime in the sst files here `/scratch1/NCEPDEV/stmp2/Guillaume.Vernieres/runs/cp1/cp1.e/COMROOT/cp1.e/gdas.20210702/00/obs/ @apchoiCMD ?

apchoiCMD commented 3 months ago

Thee Retrospective DA windows have been tested with fixed dateTime for GHRSST Ioda Converter, brief summary of test results are below;

----- For 20210702.00 ----- 212 files

5: Test     : seconds since 1981-01-01T00:00:00Z
5: Test     : obsVal:
5: Test     :     Min: -2.36087
5: Test     :     Max: 36.7395
5: Test     :     Sum: 6.38874e+07
5: Test     : obsError:
5: Test     :     Min: 0.04
5: Test     :     Max: 1.65868
5: Test     :     Sum: 636944
5: Test     : preQc:
5: Test     :     Min: 0
5: Test     :     Max: 0
5: Test     :     Sum: 0
5: Test     : longitude:
5: Test     :     Min: -179.987
5: Test     :     Max: 179.986
5: Test     :     Sum: -6.98074e+07
5: Test     : latitude:
5: Test     :     Min: -71.2478
5: Test     :     Max: 81.0295
5: Test     :     Sum: 2.26014e+06
5: Test     : datetime:
5: Test     :     Min: 1278017664
5: Test     :     Max: 1278105984
5: Test     :     Sum: 3509782890331264
----- For 20210703.00 ----- 189 files

5: Test     : seconds since 1981-01-01T00:00:00Z
5: Test     : obsVal:
5: Test     :     Min: -2.772
5: Test     :     Max: 37.5002
5: Test     :     Sum: 3.06426e+07
5: Test     : obsError:
5: Test     :     Min: 0.0100001
5: Test     :     Max: 1.31333
5: Test     :     Sum: 340891
5: Test     : preQc:
5: Test     :     Min: 0
5: Test     :     Max: 0
5: Test     :     Sum: 0
5: Test     : longitude:
5: Test     :     Min: -179.977
5: Test     :     Max: 179.984
5: Test     :     Sum: -4.45624e+07
5: Test     : latitude:
5: Test     :     Min: -71.1522
5: Test     :     Max: 81.0819
5: Test     :     Sum: 1.00678e+06
5: Test     : datetime:
5: Test     :     Min: 1278102528
5: Test     :     Max: 1278131328
5: Test     :     Sum: 1763328467194368
----- For 20210704.00 ----- 189 files

5: Test     : seconds since 1981-01-01T00:00:00Z
5: Test     : obsVal:
5: Test     :     Min: -2.304
5: Test     :     Max: 36.8412
5: Test     :     Sum: 3.08767e+07
5: Test     : obsError:
5: Test     :     Min: 0.068089
5: Test     :     Max: 1.76
5: Test     :     Sum: 346934
5: Test     : preQc:
5: Test     :     Min: 0
5: Test     :     Max: 0
5: Test     :     Sum: 0
5: Test     : longitude:
5: Test     :     Min: -179.982
5: Test     :     Max: 179.982
5: Test     :     Sum: -5.00298e+07
5: Test     : latitude:
5: Test     :     Min: -67.218
5: Test     :     Max: 81.4368
5: Test     :     Sum: -864509
5: Test     : datetime:
5: Test     :     Min: 1278189952
5: Test     :     Max: 1278216064
5: Test     :     Sum: 1788735636797184

Test results show that the difference of min and max dateTime is now within 6 hrs DA window, see the previous comparison below,

5: Test     : datetime:
5: Test     :     Min: 1123818993       <------ Problem
5: Test     :     Max: 1278103814
5: Test     :     Sum: 3509782888848048
apchoiCMD commented 3 months ago

16_Before_After