Marine obs "ex script" in the g-w

[guillaumevernieres]

Description

I understand the scope of #579 changed and so was only partially implemented. Still, the new jjob does need to do something!

Todo

Given a list of observation types (see obs_list.yaml), copy the necessary files from already prepared 6 hourly dumps to the "com obs".

Let's put the script in the g-w this time @AndrewEichmann-NOAA .

[AndrewEichmann-NOAA]

@guillaumevernieres @ShastriPaturi I discussed this with Shastri, but where are the nc4 files I'm sourcing that match up with what's in obs_list.yaml? I don't have record of it

[guillaumevernieres]

@ShastriPaturi can you paste the path to an example of cdump?

[ShastriPaturi]

/scratch1/NCEPDEV/stmp4/Shastri.Paturi/forAndrew/gdas.202107[01-02]

[AndrewEichmann-NOAA]

Observation: none of the sources in obs_list.yaml appear to match the files in forAndrew. No big deal, I should be able to work from the yamls in parm/soca/obs/config/, but something to be aware of

[AndrewEichmann-NOAA]

General question about assumptions: Can I assume that all files with obs within a window will land in the {PDY}/{cyc} directory and not in the directories for the cycle before or after? To pick a concrete example, is there any chance that the file /scratch1/NCEPDEV/stmp4/Shastri.Paturi/forAndrew/gdas.20210702/06/sst/20210702090000-OSPO-L3U_GHRSST-SSTsubskin-VIIRS_NPP-ACSPO_V2.61-v02.0-fv01.0.nc would have observations that would fall in the window for the cycle 20210702-12?

[ShastriPaturi]

@AndrewEichmann-NOAA good point. Care is being taken for not having an overlap. So, the answer is NO. If you notice an overlap, let me know. It may be a lapse on my side.

[ShastriPaturi]

@AndrewEichmann-NOAA the SSS and ADT data have been transferred to /scratch1/NCEPDEV/stmp4/Shastri.Paturi/forAndrew/

[guillaumevernieres]

Your mission @AndrewEichmann-NOAA is to copy from one place to another, and figure out the mapping between what's in the obs list and the obs "data base" file names. Don't worry about overlapping at this stage. I'm out tomorrow, but @ShastriPaturi should be able to guide you for the mapping.

[AndrewEichmann-NOAA]

@guillaumevernieres @ShastriPaturi If you want to check out what I have so far, see

scripts/exglobal_prep_ocean_obs.py
ush/soca/prep_marine_obs.py

in feature/prepocniodaobs

[ShastriPaturi]

@AndrewEichmann-NOAA can you have one class for ADT, AMSR2? The pattern for both are the same with two characters for the altimeter and the hemispheres.

[AndrewEichmann-NOAA]

@ShastriPaturi No problem

[guillaumevernieres]

@guillaumevernieres @ShastriPaturi If you want to check out what I have so far, see

scripts/exglobal_prep_ocean_obs.py
ush/soca/prep_marine_obs.py

in feature/prepocniodaobs

Over-engineered @AndrewEichmann-NOAA . That needs to be simplified. Not your fault btw, I'm the one to blame on that one. Let's tagup tomorrow and sort out this issue in a simpler way.

[AndrewEichmann-NOAA]

@ShastriPaturi What do the sst files with the labels AHI_H08, ABI_G17, and ABI_G16 correspond to? I can't find any yamls to match them

[guillaumevernieres]

@ShastriPaturi What do the sst files with the labels AHI_H08, ABI_G17, and ABI_G16 correspond to? I can't find any yamls to match them

@AndrewEichmann-NOAA , these are the geostationary satellite (Himawari and ABI). We haven't tried to assimilate these obs yet but we do need them. @guillaumevernieres We haven't tested them yet. I need to have those yamls file ready.

[AndrewEichmann-NOAA]

@guillaumevernieres Is there a particular <instrument>_<platform>code I should use for those?

[guillaumevernieres]

@guillaumevernieres Is there a particular <instrument>_<platform>code I should use for those?

Sure: instrument_platform = AHI_H08 ABI_G17 ABI_G16

[ShastriPaturi]

@guillaumevernieres Is there a particular <instrument>_<platform>code I should use for those?

Sure: instrument_platform = AHI_H08 ABI_G17 ABI_G16

@AndrewEichmann-NOAA also please note that we do not have AHI_H08 anymore in real-time. It will be AHI_H09. Also, please add a place holder for ABI_G18.

[AndrewEichmann-NOAA]

@ShastriPaturi obs sources changed and added