WCDA testing

[guillaumevernieres]

Wish list for the next WCDA test

• Create a readme or short document to describe what has been updated
• Replaces branches by hash's or tags

Ob proc for March and July/Aug 2021

• [x] Make a copy of G's obs db in shared fs
• [x] Prepare AMSR2 icec obs in ioda-v3
• [x] GDP drifters
• [x] Check sst retreivals (missing N20)
• [ ] Freeboard obs
• [ ] Check insitu profiles
• [ ] Check SSMIS North/South Ice concentration

Obs ingest and qc

• [x] Following @hyunchul386 's study, modify the QC for the sst retrievals:
  • ~Remove the distance to coast filter. Not only does it not do anything, but it's also a hack inherited from the previous system~
  • filter based on lower bound for sigo needs to be remove or adjusted
  • Somebody needs to check if our background check filter is doing what we want
  • [ ] concatenate all adt's into 1 obs space

B-matrix setup

• [ ] inflate the diagonal following what @liuxiao37k is doing
• [ ] add bump3d back as a saber block

Testing

• [ ] AC48-O500
• [ ] AC384-O025

Note from current testing

~10 days worth of cycling can be found here /scratch2/NCEPDEV/ocean/Guillaume.Vernieres/runs/iau-test/cp0

Major updates

• hybrid 3D EnVAR (30 members from @liuxiao37k)
• non-ensemble B actually reads the std. dev. instead of the variance! (bugfix)
• addition of u,v control variables
• IAU now with T, S, u & v
• ...

Issues so far

• Ice analysis is not as good as it was in golden384
• Missing Kuroshio large meander, check ADT QC
• Missing drifters, and insitu profiles
• Check currents

[ShastriPaturi]

Made a copy of G r2d2-v2-v3 in a common location: /scratch2/NCEPDEV/marineda/r2d2-v2-v3.

[ShastriPaturi]

gdp drifters SST data date is fixed and Sanity Checked and prepared for iodav3 as well. The ioda-v2 and v3 are on hera as well. /scratch2/NCEPDEV/marineda/r2d2-v2-v3/gdas_marine/ob/s2s_v1/P1D/2021-03 & 06-29 till 07-31.

@liuxiao37k the ioda-v2 for 2021-06-29 till 2021-07-31 are on orion for your hofx runs. /work2/noaa/ng-godas/r2d2/gdas_marine/ob/s2s_v1/P1D

[ShastriPaturi]

@guillaumevernieres the AMSR-2 icec for 2021-07-{01-31} & 2021-03-{01-31} in ioda-v3 on hera /scratch2/NCEPDEV/marineda/r2d2-v2-v3/gdas_marine/ob/s2s_v1/P1D/

[hyunchul386]

It would need to check and confirm that the QC's are correctly working or not in short test runs.

[guillaumevernieres]

It would need to check and confirm that the QC's are correctly working or not in short test runs.

@hyunchul386 , do you mean debugging the distance_from_coast filter or testing that the proposal of removing the 2 filters that I mention in the description is reasonable? If the later, this is the intent of the next test, we'll know after just a few cycles. If the former, gone are the days of running a full blown high-res system when one should debug at low-res with the provided ctests in either soca or the GDASApp.

[hyunchul386]

I am considering to test removing filter and checking it, just few cycles.

[AndrewEichmann-NOAA]

I would suggest tagging the global-workflow/GDASApp version, or at least noting the hashes

[guillaumevernieres]

I am considering to test removing filter and checking it, just few cycles.

Sounds good to me @hyunchul386 , remove the distance to coast and sigo check ... Did we have issues with a few obs with sigo=0 with the sst's?

[ShastriPaturi]

What is sigo @guillaumevernieres ?

[guillaumevernieres]

What is sigo @guillaumevernieres ?

... obs error.

[hyunchul386]

I am considering to start from the G-W (develop, 81c859d7) is it reasonable?

[hyunchul386]

I guess the develop of 81c859d7 is too old.

[guillaumevernieres]

I guess the develop of 81c859d7 is too old.

Sorry, I didn't realize you were planning to run with the g-w @hyunchul386 , you'll need to wait until my latest PR is merged. Why not just test with soca-science?

[hyunchul386]

Okay, I will try with soca-sci.

[hyunchul386]

• Checking the sanity for the QC filters in WCDA: Following are (o-b) of SST from METOPb at 20210720 00Z in WCDA run, without QC (upper) and with QC (middle) in Gulf Stream region

[hyunchul386]

This is the same one but globally,

[hyunchul386]

As shown in the global QC flags, more than 25% SST are rejected by the domain-check filter (mainly distance from coast). In the first two plot (with/without QC (o-b) plots), the SST (o-b)'s in the coastal region are not much abnormal, and seems that there is no reason to reject those data by the "distance from coast" filter.

[hyunchul386]

[hyunchul386]

[hyunchul386]

More evaluation plots are in https://docs.google.com/presentation/d/1dyH0pHmOawQS5vQ4kJfVb2H6m6UoKuZAFKLTW6y2Hrk/edit#slide=id.g2526a648500_0_16