This PR enables the scalar transport routines in MPAS-Atmosphere to run on GPUs through the addition of OpenACC directives. Specifically, the atm_advance_scalars_work and atm_advance_scalars_mono_work routines have been ported via OpenACC.
Additionally, timing information for OpenACC data transfers in the scalar transport routines is captured and reported in the log file by two new timers:
atm_advance_scalars [ACC_data_xfer]
atm_advance_scalars_mono [ACC_data_xfer]
To minimize data movement, the MPAS-Atmosphere dycore has also been modified to copy all invariant fields needed by the scalar transport routines onto the device before the first dynamics timestep and to delete those invariant fields after the last dynamics timestep.
This PR enables the scalar transport routines in MPAS-Atmosphere to run on GPUs through the addition of OpenACC directives. Specifically, the
atm_advance_scalars_workandatm_advance_scalars_mono_workroutines have been ported via OpenACC.Additionally, timing information for OpenACC data transfers in the scalar transport routines is captured and reported in the log file by two new timers:
atm_advance_scalars [ACC_data_xfer]atm_advance_scalars_mono [ACC_data_xfer]To minimize data movement, the MPAS-Atmosphere dycore has also been modified to copy all invariant fields needed by the scalar transport routines onto the device before the first dynamics timestep and to delete those invariant fields after the last dynamics timestep.