search

NOAA-EMC / HDASApp

GNU General Public License v3.0
0 stars 3 forks source link

Test the UFO of SATWND in 3DEnVar #2

Open JingCheng-NOAA opened 4 months ago

JingCheng-NOAA commented 4 months ago

To document the process of assimilating SATWND data in 3DEnVar.

JingCheng-NOAA commented 4 months ago

First step is creating an IODA nc format observation file from GDAS or GFS bufr file using iodaconv.

A json file is created for the GOES AMV data on Hera "/scratch1/NCEPDEV/hwrf/scrub/Jing.Cheng/jediwork/obs/test/bufr2ioda_satwind_amv_goes.json". Use script "bufr2ioda_satwnd_amv_goes.py" will create two IODA nc files in out directory:

gfs.t12z.satwnd.abi_goes-16.tm00.nc gfs.t12z.satwnd.abi_goes-17.tm00.nc

JingCheng-NOAA commented 4 months ago

The YAML file is prepared as /scratch1/NCEPDEV/hwrf/scrub/Jing.Cheng/jediwork/UFOtests/3denvar_jedi_satwnd.yaml. And the bash script to run the job is /scratch1/NCEPDEV/hwrf/scrub/Jing.Cheng/jediwork/UFOtests/sat-3dvar.sh. The executable file is changed from fv3jedi_var.x to gdas.x

Currently, the test run encountered error of YAML file parsing issue according to the log. "Assertion failed: dynamic_cast<const YAMLItemEntry*>(advance) in value, line 362 of /scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/cache/build_stage/spack-stage-eckit-1.24.5-uo33x5dafdu4ae5fl6o5ebpdurhmv7af/spack-src/src/eckit/parser/YAMLParser.cc"

JingCheng-NOAA commented 4 months ago

The parsing error is fixed by modifying the background part of the YAML file. The yaml file is readed in with no issue. However, the test stopped by complaining of missing variable: 23: Field_fail: Field wind_reduction_factor_at_10m cannot be obtained from input fields.

JingCheng-NOAA commented 3 months ago

The parsing error is fixed by modifying the background part of the YAML file. The yaml file is readed in with no issue. However, the test stopped by complaining of missing variable: 23: Field_fail: Field wind_reduction_factor_at_10m cannot be obtained from input fields.

Adding variable 'f10m' in state variable will avoid such error

JingCheng-NOAA commented 3 months ago

Adding filter Background Check will bring up the issue that observations fall out of model domain, and using LAMDomainCheck still have the same error. Exception: Assertion failed: hofx[jobs] != missing in applyFilter, line 116 of /scratch1/NCEPDEV/hwrf/save/Jing.Cheng/JEDI/GDASApp/bundle/ufo/src/ufo/filters/BackgroundCheck.cc Tried to erase key "mesh[address=0x19e4a70],halo[size=0]" from cache "NodeColumnsHaloExchangeCache" but it was not found. Erased key "mesh[address=0x19e4a70],halo[size=1]" from cache "NodeColumnsHaloExchangeCache".

JingCheng-NOAA commented 3 months ago

For testing purpose only, reprocess the IODA NC file and only select a small chunk of observation within HAFS domain to test if the experiment can run through the Background Check. Original observations orig_obs_on_domain Reduced Observations reducedmore_obs_on_domain

The analysis using Background Check is able to completed with no error with this small chunk of observations

yonghuiweng commented 3 months ago

Here is the plot when you cut off 5deg around the boundary. filterd_obs_domain_cut_5deg We can see some points around the bottom sides are still out of the domain. Here is the domain and obs by cut off 10 deg. filterd_obs_domain

JingCheng-NOAA commented 3 months ago

@yonghuiweng Thanks for the plots. It explains!

JingCheng-NOAA commented 3 months ago

OBSValue WindEsatward obs_windEastward OBSValue WindNorthward obs_windNorthward

OMB WindEastward obsomb_windEastward OMB WindNorthward obsomb_windNorthward

JingCheng-NOAA commented 3 months ago

In HAFS GSI, the assimilated SATWIND ObsTypes are all from GOES satllites: 240, 241, 245, 246, 247, 251. Besides type 241, all the types are tested in JEDI. The observation Operator YAML is recorded. ObsType 241 are GOES CIMSS AMV, currently only available after year 2023, which is beyond the testing window (20200825). There is no currently available json/python for converting this types of AMV into NC file, and no available of Observation Operator as well. This type of data will need revisit in the future.