search

PrincetonUniversity / athena

Athena++ radiation GRMHD code and adaptive mesh refinement (AMR) framework
https://www.athena-astro.app
BSD 3-Clause "New" or "Revised" License
226 stars 122 forks source link

Fix unsafe SIMD in relativistic Riemann solvers #521

Open felker opened 1 year ago

felker commented 1 year ago

Following #518 and specifically d59c678b796fb25f6da0ef769328ad384068f163, OpenMP-based explicit SIMD was disabled for frame transformation-based HLLE (Hydro and MHD) and HLLC (Hydro) relativistic Riemann solvers. This likely degrades performance; need to diagnose the root cause of the regression test failures:

    sr.hydro_convergence: failed; time elapsed: 36.8 s
    sr.hydro_shocks_hllc: failed; time elapsed: 38.1 s
    sr.hydro_shocks_hlle: failed; time elapsed: 36 s
...
    sr.mhd_shocks_hlle: failed; time elapsed: 41.1 s

And need to double check any gaps in GR/SR Riemann solver test coverage for any further correctness issues that may be lurking in the code.

c-white commented 1 year ago

If vectorization is really causing these failures, even when simdlen is eliminated, I worry about the other simd calls in the relativistic Riemann solvers:

grep '\<simd\>' src/hydro/rsolvers/**/*_rel*

They are all doing pretty much the same thing. The workhorse in most production code is hlle_mhd_rel_no_transform.cpp. At least I guess that hasn't broken, yet.

felker commented 1 year ago

yeah, I don't really understand what is wrong. Need to review what (if anything) we learned from #349 and #138