search

CliMA / OceanParameterizations.jl

Machine learning (and uncertainty quantification?) of climate model parameterizations using differentiable (and probabilistic?) programming.
MIT License
21 stars 2 forks source link

Comparing free convection parameterizations #43

Closed ali-ramadhan closed 3 years ago

ali-ramadhan commented 3 years ago

This PR compares the trained free convection NDE against existing parameterizations, namely vanilla convective adjustment, KPP, and a TKE mass flux parameterization (both from OceanTurb.jl).

All parameterizations are also compared with the LES (truth) solution.

KPP and TKE still need to be calibrated to the training simulations for a fair comparison.

Some figures

free_convection_comparisons_3

free_convection_comparisons_4

loss_matrix_plots

free_convection_nde_loss_history

ChrisRackauckas commented 3 years ago

This one doesn't look like it trained as well as some of the previous demos?

ali-ramadhan commented 3 years ago

@ChrisRackauckas Ah some previous NDEs might have been trained and tested on only one simulation so we might have just been seeing overfitting.

I think ideally we want to train on a suite of (training) simulations then test accuracy/extrapolation on a suite of (testing) simulations. I think the fact that testing accuracy gets better with training accuracy is a good sign we're not overfitting and have some chance at extrapolation.

This PR mostly focuses on comparing with existing parameterizations which can help guide whether we need more training, bigger network, etc.

ChrisRackauckas commented 3 years ago

I see. Okay, let's improve it.