search

PaulSpence / OM2-emulator

0 stars 0 forks source link

Technical and Human Resources: how do we learn to do this? #4

Open PaulSpence opened 2 months ago

PaulSpence commented 2 months ago

ML at NCI: https://opus.nci.org.au/pages/viewpage.action?pageId=320274659

https://www.nature.com/articles/s41598-024-72145-0 see also slides from Laure Zanna

Lost of great refs in https://arxiv.org/pdf/2405.18585

Where can we get the emulator code?

Other potential collabs: Andrew Shau (Hewlitt Packard Canada) Navid, Saanna (W21C), ??? WHo at NCI can help us implement the emulator code?

PaulSpence commented 2 months ago

Slides from NCI on ML: Maruf_15-Jun-23.pptx.pdf

PaulSpence commented 2 months ago

Slides from Laure Zanna at OMDP/Commodore meeting: OMDP_AI_augmented_models-1.pdf

rmholmes commented 2 months ago

I think asking Yue Sun from NCI would be a good starting point.

taimoorsohail commented 1 month ago

Hi both, thanks for sharing this idea with me. I’ve had a read of the Dheeshjith et al. 2024 (https://arxiv.org/abs/2405.18585) paper. I actually have a working neural network code (a different architecture to Dheeshjith et al., but accomplishing similar things) that does similar things that they do, but with a training data set of T, S, z, month, lat and lon from observations, which predicts S. This code (a ResNet architecture) could easily be adapted to this problem, especially given we are only working with 1-degree surface data. One thing I’ll note is that I think incorporating SSS and surface FW fluxes will be essential to getting the marginal (coldest) temps right - something the paper notes too.

I haven’t incorporated the transfer learning part in my workflow (aims 2 and 3 rely in some part on this, but it isn’t essential if we aim to just replicate most of the Dheeshjith paper, which doesn’t introduce the blended training till the end). It should be easy enough though, and just requires taking the pre-trained model, removing the top layers (which are closest to the output node), freezing the remaining layers and then incorporating new top layers based on the new training data.

Happy to try using other architectures though if you have a preference to build on the Dheeshjith work rather than use a slightly different architecture. Let me know your thoughts. BTW - I have an overleaf manuscript of the results from the ResNet work that I can share :)

rmholmes commented 1 month ago

Thanks Taimoor.

I was initially thinking we would wait for Laure's new paper that includes information from below the surface. @PaulSpence I still haven't seen that paper yet?

However, starting with what you have setup sounds like a great idea. I think both Paul and I are keen to learn ourselves, and you taking us through it might be a really efficient way to get there.

That said, I'm heading off on leave next Friday for 3 weeks, and have heaps to finish before then...

taimoorsohail commented 1 month ago

All good, I want to solidify my understanding of how the transfer learning and the time evolution bit works, so I am happy to have a play around with my code, mostly for my own learning, while you are away. Is Laure willing to share any code do you know? The arXiv pre-print is a bit light on details of the model architecture and I didn't see a data/code availability section.

I think there's scope for emulators to reproduce CMIP6 means or to produce ocean forecasts constrained by ocean observations -- so this is a handy workflow to develop locally regardless! I've found ResNet (and feed-forward NNs more generally) to work well for sparse ocean observations problems, but I know M2Lines also use CNN which is slightly different and I am less familiar with.