Open quantheory opened 3 weeks ago
@quantheory thanks for the detailed post. I edited the title to avoid confusion with the Atmosphere Driver. I will share my thoughts soon.
So, my first thought is that we should obv start from FAD, since reverse mode seems more complicated. We could start with
Step 2 may already be a can of worms, since p3 functions are already assuming a composite data type (ekat::Pack<T,N>), and I don't know if/how packs and fads will interact. I am also thinking about layout ordering when kokkos views are thrown in the equation, especially for GPU runs, where you want to keep coalesced access.
But leaving GPUs aside, I am thinking that making p3 digest both Pack and Fad types may be quite the ride...
@bartgol I should probably look into this more closely rather than try to speculate too much about the details, but my understanding is that Sacado's FAD implementation works with Kokkos by creating their own modified view implementation that has FAD support and is supposed to have a good layout. So maybe the right approach (at least in the long run) would be to add support for these views to EKAT packs, and handle Kokkos/Sacado/EKAT compatibility issues at that level as much as possible rather than in P3.
Yes, they have a specialization of views for sacado types. I'm more worried about how that plays with packs, as well as our version of team policies. The devil is always in the details. But we didn't start from scratch, that's for sure.
After talking with @bartgol at the EESM meeting last week, he suggested making a tracking issue to discuss changes that would be useful or necessary for differentiable modeling, specifically with automatic differentiation (AD). So, I'm going to make a list of needs for forward- and reverse-mode differentiation at different levels, for future reference, and then a list of applications of AD and the capabilities that I think that those applications would need.
Forward-mode SCM:
Forward-mode DP-SCREAM or global EAMxx with an idealized surface (e.g. aquaplanet):
Forward-mode E3SM with real surface models:
Reverse-mode:
Applications: