search

ECMWFCode4Earth / challenges_2020

ECMWF Summer of Weather Code 2020 challenges
49 stars 9 forks source link

Challenge #12 -Performance-portable implementation of ECRad in Julia #2

Closed EsperanzaCuartero closed 3 years ago

EsperanzaCuartero commented 4 years ago

Challenge #12 - Performance-portable implementation of ECRad in Julia

Stream 1 - Weather-related software and applications

IMPORTANT: this challenge is eligible to apply for cloud credits from the European Weather Cloud.

Goal

Performance-portable implementation of ECRad radiation package in Julia

Mentors and skills

Challenge description

While traditional compiled programming languages, like Fortran or C/C++, provide raw performance needed for HPC, modern dynamic languages like Python are often much more concise and user-friendly. The Julia programming language promises to provide the best of both worlds, providing multiple dispatch, dynamic typing and native interoperability with C and Python, while using Just-in-Time compilation (JIT) to provide performance. In addition, since compilation is delayed until runtime, Julia code can also leverage hardware-specific compilation optimizations allowing it to target CPU and GPU architectures from a single code base or with only a small amount of specialization [1].

In this project we aim to evaluate performance portability for Numerical Weather Prediction (NWP) models by porting the ECRad radiation package, a key component of the operational ECMWF model [2], to Julia. The key deliverable of this project is to develop a new Julia-based version of ECRad that can be validated against existing test cases. The performance of the port will be compared to the operational Fortran baseline, and if possible further optimized for GPUs or ARM architectures [3]. The ultimate aim of this project would be a demonstration of production-grade NWP code that is capable of targeting multiple HPC Architectures from a single code base.

References

milankl commented 4 years ago

Hey everyone, checking the IFS documentation on radiation and comparing that to what is described in the Hogan and Bozzo paper I'm a bit confused about the complexity of this project. The paper nicely summarizes the physics, but I'm wondering whether you could point me to the equations to get an idea how much time would need to be spend on the porting to Julia and how much on optimization of the Julia code & making it portable on GPUs (I assume you are thinking of approaches like GPUifyLoops.jl?)

marsdeno commented 4 years ago

Hi Milan, thanks for taking a look at our project.

Unfortunately, the IFS documentation for radiation is not up to date, and does not reference the "new" radiation package developed by Hogan et al which we are interested in, and described in the paper you link to. ECRad was developed from scratch by Robin Hogan, with clean programming in mind (attached, the user guide to the package). ecrad_documentation.pdf The JAMES paper describes scientific&algorithmic differences with respect to previous radiation models; would you like a reference describing equations relative to the previous state of the art?

We are absolutely thinking of using something like GPUifyLoops for the portability aspect. The code is used by more than one Met centre, can be used in standalone mode, and has an active research/development effort attached to it, so a portable flexible implementation of this could make quite a splash! Cheers, Olivier

mlange05 commented 4 years ago

Hi @milankl As @marsdeno mentioned, the project is purely focused on the ECRad standalone radiation code, which is much more manageable than the fully integrated version described in the IFS cycle docs. The scope of the project is also kept somewhat flexible by design, since we can pursue various interesting steps, depending on how things go (this is experimental stuff, after all :wink: ). On the one hand, having a complete, validated port of the code in a "high-level language" would be of immense value for the community and future research efforts, while porting only one of the solvers should already give us enough material to do some interesting performance and GPU porting experiments. And yes, GPUifyLoops.jl is exactly the sort of approach we want to look into here. :smile:

milankl commented 4 years ago

Thanks a lot for the input! I was wondering whether you also plan to make use of Julia's type-flexibility. Technically one could write an entire ECRadiation.jl (Yes, I see you always want to keep things as short as possible, but honestly, no one outside of the radiation community knows that Rad stands for radiation...) package that can be compiled on the fly for any number type. In ShallowWaters.jl for example, you can run the entire model in 16bit, but you can also keep the prognostic variables at 32bit, use Float16/BFloat16 for the tendencies and do MPI-like communication in 8bit. All you need to do is to change some input arguments. Depending on the hardware, this is at the moment not necessarily faster, but allows you to directly exploit the speed-up of reduced precision when ported to GPU/TPU-like hardware. I don't know what the reduced precision potential of ECRad is, but I'm sure if you want to go in that direction you should consider it right from the start. There is no performance-cost for writing ECRadiation.jl in a type-flexible way but using it only with standard types like Float32/Float64.

milankl commented 4 years ago

Also, I was wondering whether you have thought of advertising this challenge on Julia discourse and/or Julia slack?

marsdeno commented 4 years ago

Sorry for slow response, actively fighting bugs in our code! So,

natgeo-wong commented 4 years ago

Thank you @milankl for advertising this in the Julia Slack! I would be interested in participating as well! Are there sample codes for us to gauge?

milankl commented 4 years ago

@marsdeno @mlange05 I assume you are aware of RRTMGP.jl. How does this radiation scheme differ from ECRadiation or how much of their code could be reused in this project?

natgeo-wong commented 4 years ago

Hi @marsdeno @mlange05, just checking in are there any news/updates or sample codes for us to gauge?

jwagemann commented 4 years ago

Join us for our LIVE @ecmwf Summer of Weather Code Ask Me Anything session on 1 April 2020 at 2 pm (CET) (tomorrow).

Get infos first hand from the #ESoWC2020 organisers, mentors and former #ESoWC participants. ➡️Sign up

marsdeno commented 4 years ago

Hi @milankl , sorry for missing your post! Indeed we are aware of rrtmpg.jl, and believe it could absolutely be used as a template /strong inspiration for this work. The ecrad radiation scheme differs from rrtmg in algorithmic and implementation choices, with in particular an accent on encapsulating the four main components (gas optics, aerosol optics cloud optics and solver) allowing easy substitution/addition of new implementations. An additional link to a presentation of the scheme is below : https://www.ecmwf.int/en/elibrary/16901-ecrad-new-radiation-scheme-ifs

marsdeno commented 4 years ago

Hi @natgeo-wong unfortunately, distribution of the code is covered by a license precluding us from pasting code examples online, I'm very sorry (and aware of how frustrating this is!) I can tell you that it is clean, modular "modern fortran", composed of 250 files, with around 60k loc total. Hope that helps a little!

jwagemann commented 4 years ago

Only 4 days left to apply to be part of ECMWF Summer of Weather Code 2020. Application deadline: Wednesday, 22 April 2020 at 23:59 (BST). Submit your proposal here.

milankl commented 4 years ago

Just to let you know @marsdeno and @mlange05, I've decided against this project due to time constraints. Challenge #14 (#4) fits better into my PhD such that I decided to write a proposal for this instead. In case anyone wants to push Julia for NWP in the future, please let me know, I would like to contribute wherever I can.

marsdeno commented 4 years ago

all the best for your PhD, @milankl !