JeffersonLab / chroma

The Chroma Software System for Lattice QCD
http://jeffersonlab.github.io/chroma
Other
57 stars 51 forks source link

QMP Thread safety level in QDP++ #2

Closed bjoo closed 2 years ago

bjoo commented 11 years ago

QMP_initialize allows us to request a thread safety level (QMP_THREAD_SINGLE, QMP_THREAD_FUNNELED, QMP_THREAD_SERIALIZED, QMP_THREAD_MULTIPLE) mimicking the levels offered by MPI. A recent commit of QMP ( commit 9cdf6875d0e7077cf3df2e8a5b56f26646302e52 ) performs the MPI Init properly requiring the corresponding MPI thread safety level (before it was always MPI_THREAD_SINGLE==QMP_THREAD_SINGLE).

QDP++ should call this init function 'properly', with the right thread level. Non threaded code should use QMP_THREAD_SINGLE, and threaded code may need higher levels. Right now probably our master thread + fork join, may indicate QMP_THREAD_FUNNELED as the right choice, but if we intend to use Chroma with multi-threaded libraries, where individual threads can make MPI calls (you never know) it may be worth to go for QMP_THREAD_MULTIPLE (=> MPI_THREAD_MULTIPLE).

This may use code paths in MPI with higher overhead than MPI_THREAD_SINGLE. In terms of code we could choose to call QMP/MPI_THREAD_SINGLE, or QMP/MP_THREAD_MULTIPLE based on how many threads there are (qdpNumThreads()).