No vertical localization option in LETKF

[SamuelDegelia-NOAA]

As discussed at today's joint meeting, there is currently no option for vertical localization in JEDI-based LETKF. You can see more details here. The only way to use vertical localization right now in JEDI is to use the Gain form (GETKF) solver which uses modulated ensembles to emulate model-space vertical localization. The GETKF expands the ensemble to accomplish this vertical localization which significantly increases the cost. I performed some tests with our FV3-JEDI case and found ~30x time cost to run GETKF compared to LETKF.

The Lei et al. (2018) paper that the JEDI-based GETKF is derived from describes how the size of the expanded ensemble is proportional to the vertical localization length scale and the fraction of retained variance (which is a parameter you can set in JEDI):

The size of the expanded ensemble is proportional to the rank of the vertical localization matrix—for a vertical localization scale of 1.5 (3.0) scale heights, 12 (7) eigenvectors explain 96% of the variance of the localization matrix, so the expanded ensemble is 12 (7) times larger than the raw ensemble.

This suggests that tuning the GETKF localization parameters could speed up the solver. Once I get our new MPAS-JEDI case working with LETKF, I plan to run tests with GETKF to explore how these parameters impact efficiency.

Here is an example section of the yaml file which sets the GETKF and its associated parameters:

local ensemble DA:
  solver: GETKF
  vertical localization:
    fraction of retained variance: .95
    lengthscale: 1.5
    lengthscale units: logP

[guoqing-noaa]

@SamuelDegelia-NOAA Could you try the obs space vertical localization in LETKF? These are some examples I got from NCAR:

[SamuelDegelia-NOAA]

@guoqing-noaa You are right, it looks like this functionality was added. I apologize for the confusion here. Apparently the documentation I linked above has not been updated in 2 years. Really sorry about that.

I tried using the yaml sections you sent me but they did not work. Now all I see are zero increments throughout the profile. But at least I am aware that this functionality exists now and can work on debugging.

[SamuelDegelia-NOAA]

I'm going to close this issue since the localization in LETKF is better understood now. I clearly need to find some better documentation for EnKF in JEDI. Thanks for the help @guoqing-noaa.

[guoqing-noaa]

@SamuelDegelia-NOAA Thanks for your testing. It is good to document this.