NOAA-EMC / GDASApp

Global Data Assimilation System Application
GNU Lesser General Public License v2.1
14 stars 28 forks source link

Document all python api converter work for GPS-RO bufr. #648

Closed nicholasesposito closed 5 months ago

nicholasesposito commented 9 months ago

This issue will document decisions and work made for GPS-RO bufr for the python api The tasks:

nicholasesposito commented 9 months ago

List of SAIDs in the bufr file and satellite they are. 3 MetOpB 4 MetOpA   5 MetOpC 42 TerraSAR-X 43 (not in current file) TanDEM-X 44 PAZ 265 GeoOptics CICERO OP1 750 COSMIC-2 Equatorial       6 satellites 751 752 753 754 755 803 GRACE C (see Note a few comments later) 825 KOMPSAT-5


The GSI dictates that we need the variables below. The ending string in each of the columns is the array from GSI that each variable is associated with.

  data hdr1a / 'YEAR MNTH DAYS HOUR MINU PCCF ELRC SAID PTID GEODU' / 
  data nemo /'QFRO'/

data2a -> ROSEQ3 data1b -> ROSEQ1    

              if (ref_obs) then
                 cdata_all(1,ndata) = ref_error      ! gps ref obs error (units of N)       data2a(4,k)
                 cdata_all(4,ndata) = height         ! geometric height above geoid (m)     data2a(1,k)
                 cdata_all(5,ndata) = ref            ! refractivity obs (units of N)        data2a(2,k)
!                cdata_all(9,ndata) = ref_pccf       ! per cent confidence (%)              data2a(6.k) (don't need but added)
              else
                 cdata_all(1,ndata) = bend_error     ! gps bending error (radians)          data2a(4,k)
                 cdata_all(4,ndata) = impact         ! impact parameter (m)                 data2a(1,k)
                 cdata_all(5,ndata) = bend           ! bending angle obs (radians)          data2a(2,k)
!                cdata_all(9,ndata) = bend_pccf      ! per cent confidence (%)              data2a(6.k) (don't need but added)
              endif
              cdata_all(9,ndata) = pcc             ! profile per cent confidence (0 or 100)     bfr1ahdr(6)
              cdata_all(2,ndata) = dlon            ! grid relative longitude                   dlon = dlon_earth = rlon * deg2rad = data1b(2,k) * deg2rad     (deg2rad in constants = pi/180.0_r_kind)
              cdata_all(3,ndata) = dlat            ! grid relative latitude                    dlat = dlat_earth = rlat * deg2rad = data1b(1,k) * deg2rad     (deg2rad = pi/180.0_r_kind)
              cdata_all(6,ndata) = t4dv            ! time relative to analysis (hour)          
              cdata_all(7,ndata) = ikx             ! type assigned to ref data                 ??? from convinfo --- ignore
              cdata_all(8,ndata) = nmrecs          ! profile number                            ??? Not added for now
              cdata_all(10,ndata)= roc             ! local radius of curvature (m)             bfr1ahdr(7)
              cdata_all(11,ndata)= said            ! satellite identifier                      bfr1ahdr(8)
              cdata_all(12,ndata)= ptid            ! platform transmitter id number            bfr1ahdr(9)
              cdata_all(13,ndata)= usage           ! usage parameter                           from convinvo (ignore for now)
              cdata_all(14,ndata)= dlon_earth_deg  ! earth relative longitude (degrees)        data1b(2,k
              cdata_all(15,ndata)= dlat_earth_deg  ! earth relative latitude (degrees)         data1b(1,k)
              cdata_all(16,ndata)= geoid           ! geoid undulation (m)                      bfr1ahdr(10)

which when semi-combined results in this:

  3d  int64   NC003010/ROSEQ1/ROSEQ2/MEFR      MetaData/frequency
  3d  float   NC003010/ROSEQ1/ROSEQ2/IMPP     MetaData/impactParameterRO (m)
  3d  float   NC003010/ROSEQ1/ROSEQ2/BNDA[1]  ObsValue/bendingAngle (radians)   
  3d  int     NC003010/ROSEQ1/ROSEQ2/FOST[1]     (not saved)
  3d  float   NC003010/ROSEQ1/ROSEQ2/BNDA[2]  ObsError/bendingAngle (radians)
  3d  int     NC003010/ROSEQ1/ROSEQ2/FOST[2]   (not saved)
  2d  int     NC003010/ROSEQ1/PCCF             bend_pccf    %
  2d  int     NC003010/ROSEQ3/HEIT             MetaData/height       (m)
  2d  float   NC003010/ROSEQ3/ARFR[1]          MetaData/atmosphericRefractivity   N-units) 
  2d  int     NC003010/ROSEQ3/FOST[1]     (not saved)
  2d  float   NC003010/ROSEQ3/ARFR[2]          ObsError/atmosphericRefractivity N-units
  2d  int     NC003010/ROSEQ3/FOST[2]     (not saved)
  2d  int     NC003010/ROSEQ3/PCCF             ref_pccf MetaData/percentconfidence

During validation, there were some things that came up: qualityFlags in file I was given: 0,64,8256, etc. In mine, it's the same. It's interesting because the bufr documention says it should be 0-16, but even in the bufr dump file, the values there are the same values I had so I didn't touch it.

satelliteAscendingFlag in file i was given: 0,1. in mine, it's qfro One of the GSI scripts says the file has to do with what the qfro says. I decided to keep it that way and let UFO deal with it.

platformTransmitterId became satelliteTransmitterId

ObsError/bendingAngle. There are values in my file but it is _ in the file I was given to compare to. I decided to leave it as there is ObsError work that will be added in UFO.

nicholasesposito commented 9 months ago

Two python api scripts were written. One that outputs into one large file, and one that outputs each satellite into its own output file. The files are in GDASApp and the branches are: feature/NickE_gpsro_bufr feature/NickE_gpsro_bufr_split

There are no print statements in either, and the logger has been added for when we need information about the data. Both python scripts use JSON files.

nicholasesposito commented 9 months ago

Note that the Grace satellite (SAID == 803) is currently nowhere in GSI. We decided to keep it in the output as we will likely use it in the future.

nicholasesposito commented 5 months ago

I'm closing this because the initial GPS-RO converter has been merge. Xuanli is continuing to make changes as needed.