Feature/openmp - Githubissues

lattice / quda

QUDA is a library for performing calculations in lattice QCD on GPUs.

https://lattice.github.io/quda

Other

287 stars 94 forks source link

Feature/openmp #1411

Closed maddyscientist closed 9 months ago

maddyscientist commented 10 months ago

This is mainly a quality of life PR, as well as adding some building work for future work:

Properly enable OpenMP host compilation
OMP parallelization of some of the tests codes (gauge force/path, hisq force, fat/long link construction, staggered dslash reference)
Acceleration and robustness improvement of (staggered_)dslash_ctest; we now use the same gauge field across different partitioning
- this has an especially dramatic improvement on the execution time for the improved staggered fermion tests when constructing the fat/long links
some sundry cleanup resulting from the above additions

mathiaswagner commented 10 months ago

Is QUDA_OPENMP exposed in cmake ?

maddyscientist commented 10 months ago

Is QUDA_OPENMP exposed in cmake ?

Yes, it has been for about 5 years 🙂

maddyscientist commented 10 months ago

Latest few commits fix a couple of additional issues I found in testing:

QUDA_CTEST_LAUNCH needed an additional FORCE to enable the user to override the setting, e.g., to run ctest on different process counts without nuking the build
For the dslash ctests, make the host quark field creation a one-time event, created when the host gauge field is created. This reduces the test overhead, and ensures we use the random source field for all partitions and precision.

Also noting that the improvements from this PR reduce the ctest overhead on the CSCS CI by over 100 seconds, so a welcome boost to CI throughput.

mathiaswagner commented 10 months ago

@Jenkins test this please.

mathiaswagner commented 10 months ago

@jenkins test this please.

mathiaswagner commented 10 months ago

@jenkins test this please.

weinbe2 commented 10 months ago

As a thought, in the spirit of the raw_pointer function for the raw gauge field pointer, it'd also be nice to have a raw_pointer_ghost (or whatnot) that returns a void** for CPU-ordered fields

mathiaswagner commented 10 months ago

Is QUDA_OPENMP exposed in cmake ?

Yes, it has been for about 5 years 🙂

Guess it is one of the options I never use. Should we turn it ON by default? We could also rely on cmake to detect support and make QUDA_OPENMP=ON mean request OpenMP if supported by the compiler.

maddyscientist commented 10 months ago

Is QUDA_OPENMP exposed in cmake ?

Yes, it has been for about 5 years 🙂

Guess it is one of the options I never use. Should we turn it ON by default? We could also rely on cmake to detect support and make QUDA_OPENMP=ON mean request OpenMP if supported by the compiler.

I didn't turn it on by default since I still consider it very experimental. In particular Eigen has huge slowdowns with OMP enabled, and until we work that out, I don't think we should enable by default.

weinbe2 commented 9 months ago

Is QUDA_OPENMP exposed in cmake ?

Yes, it has been for about 5 years 🙂

Guess it is one of the options I never use. Should we turn it ON by default? We could also rely on cmake to detect support and make QUDA_OPENMP=ON mean request OpenMP if supported by the compiler.

I didn't turn it on by default since I still consider it very experimental. In particular Eigen has huge slowdowns with OMP enabled, and until we work that out, I don't think we should enable by default.

I can't test it right now, but it looks like there's a compile-time define EIGEN_DONT_PARALLELIZE (circa 2019...) that can be used to disable OpenMP. Source: https://stackoverflow.com/a/56613641