search

sandialabs / Albany

Sandia National Laboratories' Albany multiphysics code
Other
276 stars 89 forks source link

Port responses to kokkos for uvm-free access #1049

Closed mcarlson801 closed 4 months ago

mcarlson801 commented 4 months ago

This PR updates ResponseGLFlux and ResponseSurfaceVelocityMismatch so that they can be used in uvm-off GPU builds. This includes porting a number of evaluators for scattering and communicating responses to kokkos for device-friendly access.

Additionally, this PR also fixes some existing kokkos ported evaluators to avoid some unintended host accesses that showed up in a uvm-off build. I've also added evaluator names to as many of the parallel_for regions as I could find to aid in performance profiling and debugging.

After this PR is merged, the following tests will now be passing in a uvm-off build:

landIce_FO_GIS_ForwardSensitivity
landIce_FO_GIS_Unstructured
landIce_FO_GIS_Unstructured_Restart
landIce_FO_GIS_Unstructured_Memoization
landIce_FO_GIS_Unstructured_FROSch
landIce_FO_GIS_Manifold
landIce_FO_MMS_FO_GLF_TRI_CORNER_CASE
landIce_FO_MMS_FO_GLF_QUAD
landIce_FO_MMS_FO_GLF_TRI_NONCONST_VELOCITY
landIce_FO_MMS_FO_GLF_TRI
landIce_FO_Thermo_Dry_Bed
landIce_FO_Thermo_Wet_Bed
landIce_FO_Thermo_Wet_Bed_FEA
landIce_FO_Thermo_Wet_Bed_FROSch
landIce_Enthalpy_Humboldt
landIce_Enthalpy_Kleiner_A
landIce_Enthalpy_Kleiner_B

The next target for the uvm-free porting process is to port L2ProjectedBoundaryLaplacian to kokkos which will get a large number of additional tests to start passing. (I think this will get most everything remaining in FO_GIS to pass)

Finally, I've re-enabled all of the cuda tests for the uvm-free build and I will instead start an issue to track the status of which tests and expected to pass/fail with uvm disabled.