Open mjlosch opened 2 years ago
We can add these non-TAF experiments:
OpenAD.tap_adj: grdchk summary : RMS of 8 ratios = 3.3808379600186E+01
OpenAD.oad: grdchk summary : RMS of 8 ratios = 3.3808379724557E+01
OpenAD.oad.kpp: grdchk summary : RMS of 8 ratios = 3.3808313630796E+01
OpenAD.oad.ggl90: grdchk summary : RMS of 8 ratios = 3.5426055083717E-01
global_ocean.90x40x15.oad: grdchk summary : RMS of 4 ratios = 5.9224477025747E-02
global_with_exf.tap: grdchk summary : RMS of 5 ratios = 2.6722023344107E-04
Once #748 is merged, only these experiments with RMS values of >1e-4
remain:
lab_sea
OpenAD.oad.ggl90
global_ocean.cs32x15.seaice
global_ocean.cs32x15.seaice_dynmix
These involve either lab_sea or ggl90, two packages with potentially inaccurate gradients.
For the OpenAD.oad.ggl90
experiment, I can reduce the gradient error RMS value to 9e-5
when I use mxlMaxFlag=1
(and even to 9e-6
after adjusting grdchk_eps=1.e-4
) with TAF-AD, which is consistent with previous experience that the mxlMaxFlag=2
setting is even unstable and leads to blow-up of the AD-model. With OpenAD code, the RMS value remains high, so here the OpenAD code may have a problem. Maybe we should add a ggl90-check for TAF-AD code.
The RMS value for global_ocean.cs32x15.seaice_dynmix
can be reduced a little to 8e-4
by using grdchk_eps=1e-1
, but at the cost of increasing the RMS value of the main experiment to >1e-4
, so that's probably not a good solution.
I am listing the experiments with grdchk RMS of
> 1e-4
:At least some of these numbers can be improved very much by adjusting the
grdchk_eps
, e.g. for OpenAD settinggrdchk_eps=1e-6
(currently 1e-2) reduces the RMS to 3.5E-01 (still not great), for global_ocean.90x40x15, settinggrdchk_eps=1e-4
(currently 1e-2), reduces the RMS to 2.8E-06, and so on. Some cannot be improved and the AD gradients are probably wrong, but it is difficult to identify those, before we do not fix the ones where the large RMS are due to inappropriategrdchk_eps
.I suggest to adjust these individual
grdchk_eps
to get a better agreement, where possible. This implies updating the output, as the FD numbers will change.