Minimize the seasonal TEC bias by revising the eddy mixing parameterization

[ZhuxiaoLi66]

This project is mainly aiming to minimize the operational seasonal TEC bias through the revision of the eddy mixing parameterization in WAM. At the same time, we will also keep a close eye on the influence of parameterization revision on the change of the neutral density with some validation. The bias improvement also includes estimating the TEC change due to the day-number bug fix in WAM.
The output of the operational WAM-IPE and that of several designed free WAM-IPE coupling annual runs for the period about 20210715 - 20220705, also the output of the operational GloTEC during the same year-long period will be applied to do the diagnostics and mathematic analysis for parameterization change.

[ZhuxiaoLi66]

Here are some project history & experience here for future applications & notes.

1. Since there are no drivers on Hera for this recent year-long period, it took me quite some time to download the right data from HPSS to Hera and learn how to make the model driver files from the original files with the help of Adam. Now the run script for WAM-IPE free coupling and the drivers folder is built up very well.
2. Also got the lesson that we can't use some of the operational WAM output as the simulation IC files directly, Since, after the weighting of some extreme abnormal observational values, the output becomes numerically & physically unbalanced, and will cause the runs crash at a certain time after quite a period, sometimes takes 2 or 3 weeks of the simulation time length to crash. it is better to just use the output before data assimilation.

Current runs situation & history:

1. Downloaded the related 2D output of year-long oprational WAM-IPE from HPSS.
2. There are two WAM-IPE coupling free runs that have been completed for 9 months since 20210715. a) the run with the same setting of the current operational version. includes the dayno bug & keddy0=140. b) the run with the same setting except getting the dayno bug fixed. Keddy0=140.
3. The two final effective annual runs started on Sep. 21, which means it takes 17 days to complete 9 months of simulation time length for the annual coupling simulations.

[ZhuxiaoLi66]

We tested different eddy mixing parameterization sets as following,

1. original: keddy0=140, keddy_semiann= 60
2. para1: keddy0=80, keddy_semiann= 20
3. para2: keddy0=40, keddy_semiann= 0
4. para3: keddy0=60, keddy_semiann= 0
5. para4: keddy0=70, keddy_semiann= 0

Based on many comparisons of the TEC bias ratio against GloTEC of the annual runs with the different parameterization sets, we finally choose para3 whose bias ratio time serious is the closest to the ideal value of 1.0. The following are two plots for the latest update of the TEC bias ratio.

[ZhuxiaoLi66]

In the meantime, we verified the performance of the new parameter set on neutral density for 2015 St. Patrick storm. We do validation of the WAM neutral density along the GRACE satellite orbits during the storm time.
The results are very encouraging, the storm run with the new parameter set & observational driver input have a better match with GRACE data at storm peak time against former parameterization.

[ZhuxiaoLi66]

Here the plot also includes the performance of the WAM run with new parameters in the newBz SW driver-derived mode.

[ZhuxiaoLi66]

[ZhuxiaoLi66]

[twfang]

@ZhuxiaoLi66 Thanks for the plots. Moving forward, could we modify the legend so that it says "operational run with dayno_fix" instead of "dayno_fix", so that we remember what are these runs? Glad to see that the testing is going well. Thanks!

[ZhuxiaoLi66]

@twfang this run is not an operational run, we could do 'keddy0=140 & keddy_semiann=60' to remind us for later review. since the dayno_fix applies for both of the runs, maybe no need to mention it here.

[ZhuxiaoLi66]

The following is the update of the para3 run which extends to 20220710 now. so far so good.

[ZhuxiaoLi66]

The para3 (keddy0=60) annual run is extended to 20221226, almost done. The following is the update of its bias ratio plot. Looks like we have gotten kind of high values at the end of August and the beginning of September.

[ZhuxiaoLi66]

[ZhuxiaoLi66]

In general, it is a little bit high, while looks fine.

[timfullerrowell]

Fascinating. The semi-annual is emerging. Looks like we need a small semi-annual with a peak at equinox. Together with a slight reduction in the mean.

On Thu, Feb 9, 2023 at 3:38 PM ZhuxiaoLi @.***> wrote:

In general, it is a little bit high, while looks fine.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-SWPC/GSMWAM-IPE/issues/69#issuecomment-1424931343, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5BFOHMHXNYTIGPFA2F2KTWWVWXDANCNFSM6AAAAAAQ76SLF4 . You are receiving this because you were assigned.Message ID: @.***>

[ZhuxiaoLi66]

Thanks, Tim. It seems the semi-annual peaks are already at the equinox, are you sure what we want to add more? maybe in the opposite way? Please correct me if I am wrong.

[ZhuxiaoLi66]

maybe we can use a minus sign for the peak.

[akubaryk]

dev run on WCOSS2 getting keddy0=70 and keddy_semiann=0 from cycle 2023021618

outputs on HPSS: /NCEPDEV/swpc-para/1year/swpc.wam/WCOSS2/upgrade

[ZhuxiaoLi66]

Got it, thanks, Adam.

[ZhuxiaoLi66]

A new eddy mixing ratio parameter set (keddy0=70 and keddy_semiann=-10) has been designed for a new annual test WAM-IPE run which aims to get a better TEC output bias ratio against GloTEC data.

[ZhuxiaoLi66]

The following is the performance comparison among para5 (keddy0=70 and keddy_semiann=-10), para4 (keddy0=70 and keddy_semiann=0)) and para3 (keddy0=60 and keddy_semiann=0). Seem para5 works well and decrease the semiannual amplitude as expected so far.

[ZhuxiaoLi66]

the following is the update of the bias ratios for para5, and para4 annual runs.

[ZhuxiaoLi66]

This plot shows the comparison of para4 (keddy0=70) and para3 (keddy0=60) only with further extended time.

[ZhuxiaoLi66]

The following bias ratio plot is a comparison between the latest eddy mixing coefficient test (keddy0=70, keddy_semiann=-10) annual run and the run with the original eddy mixing parameters (keddy0=140, keddy_semiann=60).

[ZhuxiaoLi66]

With mean, stddev, and RMSE from 1.0 base.

[ZhuxiaoLi66]

A new parameterization run with setting keddy_semiann to -15 against the value of -10 is running, aiming to further mitigate the slight seasonal amplitude in the above bias ratio plot for -10 run. so far we have gotten output for the period of 20211101-20220408. The results are not as expected, it seems the new parameter makes the spikes bigger, both at around equinox and some other points as shown in the plot below. this might due to the big variation amplitude, or the complicated response of the physics to the parameterization.

[ZhuxiaoLi66]

the run also get a bigger rmse and std, about 0.11 against 0.10 for -10 run.

[ZhuxiaoLi66]

The annual run with the new parameterization of skeddy0=70, skeddy_semiann=-10, has been extended to the end of Apr. 2023. The extended run shows a lower bias ratio against GloTEC on average from Mar to Apr of 2023 than the same period of 2022. This explains the recent (from Mar) persistent low bias ratio for operational WAM-IPE output in plots of Daily_TEC_bias_ratio_WAM-IPE/GloTEC.

[ZhuxiaoLi66]

[ZhuxiaoLi66]

[ZhuxiaoLi66]

[ZhuxiaoLi66]

The plot below shows the comparison of the TEC bias ratio against GloTEC between recent operational WAM-IPE output and WAM-IPE freerun with the new Keddy parameters. The period of Apr.14-Oct.1 2023 is time after applying the new parameters in the operational runs.

[ZhuxiaoLi66]

[timfullerrowell]

It's like the operational run has not picked up the new values?? Is that possible?

On Wed, Nov 8, 2023 at 1:57 PM ZhuxiaoLi @.***> wrote:

[image: bias_ratio_opr_wfs_wrs_para5_20230414_1001] https://user-images.githubusercontent.com/22546571/281541059-03ee6744-3b35-4e01-a5be-c54aadb6a302.png

— Reply to this email directly, view it on GitHub https://github.com/NOAA-SWPC/GSMWAM-IPE/issues/69#issuecomment-1802662356, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5BFOATPU7ILY72IOFR3KDYDPWZRAVCNFSM6AAAAAAQ76SLF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBSGY3DEMZVGY . You are receiving this because you were assigned.Message ID: @.***>

[akubaryk]

No, this is the relevant namelist for the 2023/11/08 18Z WFS forecast that is currently running.

     wam_climate=.false., wam_swpc_3day=.true., wam_cires_rdata=.false.,
     wam_swin = .true.,
     wam_smin=.false., wam_smax=.false., wam_saver=.true., wam_geostorm=.false.,
     wam_gwphys=.false., wam_solar_in=.false., wam_ion_in=.false.,
     wam_das_in=.false., wam_smet_in=.false., wam_netcdf_inout=.true.,
     wam_tides_diag=.false., wam_pws_diag=.false., wam_gws_diag=.false.,
     jh0=1.75, jh_tanh=0.5, jh_semiann=0.5, jh_ann=0.0, jh_st0=25000.0, jh_st1=5000.0,
     skeddy0=70.0, skeddy_semiann=-10.0, skeddy_ann=0.0,
     tkeddy0=280.0, tkeddy_semiann=0.0, tkeddy_ann=0.0
/

[ZhuxiaoLi66]

I checked the ratio values, they are very close, but not exactly the same, also given they are global mean values. We could say that the wrs runs are different from the corresponding wfs runs by different IC files, but output very close to the latter, especially at the global mean value level.

[ZhuxiaoLi66]

Tim, if you mean picking up the new parameters for the operational runs, I think Adam gave the answer.

[timfullerrowell]

Thanks Adam, It is surprising that with and without DA is so big. All the other comparisons we have made it is <~5% for globally and UT averaged quantities Any thoughts, anyone? Tim

On Wed, Nov 8, 2023 at 3:28 PM akubaryk @.***> wrote:

No, this is the relevant namelist for the 2023/11/08 18Z WFS forecast that is currently running.

 wam_climate=.false., wam_swpc_3day=.true., wam_cires_rdata=.false.,
 wam_swin = .true.,
 wam_smin=.false., wam_smax=.false., wam_saver=.true., wam_geostorm=.false.,
 wam_gwphys=.false., wam_solar_in=.false., wam_ion_in=.false.,
 wam_das_in=.false., wam_smet_in=.false., wam_netcdf_inout=.true.,
 wam_tides_diag=.false., wam_pws_diag=.false., wam_gws_diag=.false.,
 jh0=1.75, jh_tanh=0.5, jh_semiann=0.5, jh_ann=0.0, jh_st0=25000.0, jh_st1=5000.0,
 skeddy0=70.0, skeddy_semiann=-10.0, skeddy_ann=0.0,
 tkeddy0=280.0, tkeddy_semiann=0.0, tkeddy_ann=0.0

/

— Reply to this email directly, view it on GitHub https://github.com/NOAA-SWPC/GSMWAM-IPE/issues/69#issuecomment-1802780412, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5BFOB3NK2YLFXVNPEN523YDQBR7AVCNFSM6AAAAAAQ76SLF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBSG44DANBRGI . You are receiving this because you were assigned.Message ID: @.***>

[ZhuxiaoLi66]

Below is the plot same with above but add the latest October output, it seems we still running low in TEC against GloTEC with the new parameters and recent drivers...

[ZhuxiaoLi66]

The PPT below is a summary of the recent investigation on the low bias-ratios of WAM-IPE against GloTEC in 2023 than 2022 with the new eddy-mixing parameterizations.

[Uploading Bias_ratio_low_in 2023.pptx…]()

[ZhuxiaoLi66]

I am sorry, everyone. It seems that the uploading of the PPT in my home machine didn't succeed. I have to retry this after I go back home this afternoon. Sorry about this!