search

StevePny / DataAssimBench-Examples

Examples for the DataAssimBench benchmarking package for applying AI/ML and data assimilation
Apache License 2.0
1 stars 1 forks source link

How to access https://github.com/StevePny/DataAssimBench? #6

Closed wuxinwang1997 closed 2 weeks ago

wuxinwang1997 commented 1 year ago

Hello,

I am interested in the benchmark library used in this GitHub project. I noticed that your code seems to be using DataAssimBench (https://github.com/StevePny/DataAssimBench), but I am currently unable to access it. Would it be possible for you to provide some information about how you are using it and how it has helped you in your project, or perhaps suggest an alternative way for me to access the library? I am curious to learn more about this library and would appreciate any insights you can provide.

Thank you!

StevePny commented 2 weeks ago

Hi @wuxinwang1997 the DataAssimBench is now public: https://github.com/StevePny/DataAssimBench

We are preparing a publication to describe its use and potential applications. An example application is given by Solvik, Penny, and Hoyer (2024):

4D-Var using Hessian approximation and backpropagation applied to automatically-differentiable numerical and machine learning models

Constraining a numerical weather prediction (NWP) model with observations via 4D variational (4D-Var) data assimilation is often difficult to implement in practice due to the need to develop and maintain a software-based tangent linear model and adjoint model. One of the most common 4D-Var algorithms uses an incremental update procedure, which has been shown to be an approximation of the Gauss-Newton method. Here we demonstrate that when using a forecast model that supports automatic differentiation, an efficient and in some cases more accurate alternative approximation of the Gauss-Newton method can be applied by combining backpropagation of errors with Hessian approximation. This approach can be used with either a conventional numerical model implemented within a software framework that supports automatic differentiation, or a machine learning (ML) based surrogate model. We test the new approach on a variety of Lorenz-96 and quasi-geostrophic models. The results indicate potential for a deeper integration of modeling, data assimilation, and new technologies in a next-generation of operational forecast systems that leverage weather models designed to support automatic differentiation.