search

JuliaParallel / MPI.jl

MPI wrappers for Julia
https://juliaparallel.org/MPI.jl/
The Unlicense
379 stars 122 forks source link

`MPI.has_cuda()` fails on Spectrum MPI #357

Closed vchuravy closed 4 years ago

vchuravy commented 4 years ago

using the test script

using MPI
MPI.Init()

@show MPI.has_cuda()

fails with

ERROR: ERROR: ERROR: ERROR: LoadError: LoadError: LoadError: LoadError: ccall: could not find function MPIX_Query_cuda_support in library /opt/ibm/spectrum_mpi/lib/libmpi_ibm.soccall: could not find function MPIX_Query_cuda_support in library
/opt/ibm/spectrum_mpi/lib/libmpi_ibm.soccall: could not find function MPIX_Query_cuda_support in library /opt/ibm/spectrum_mpi/lib/libmpi_ibm.so
Stacktrace:
Stacktrace:
Stacktrace:ccall: could not find function MPIX_Query_cuda_support in library /opt/ibm/spectrum_mpi/lib/libmpi_ibm.so
Stacktrace:
 [1]
simonbyrne commented 4 years ago

What does MPI.MPI_LIBRARY_VERSION give?

simonbyrne commented 4 years ago

Do you need to rebuild MPI.jl?

vchuravy commented 4 years ago

MPI.MPI_LIBRARY_VERSION = "Open MPI v10.3.0.00rtm0, package: IBM Spectrum MPI, ident: IBM Spectrum MPI, repo rev: IBM_SPECTRUM_MPI_10.03.00.00_2019.04.04_RTM0, Unreleased developer copy\0"

vchuravy commented 4 years ago

I just found this entertaining tibit (from https://hpc.llnl.gov/training/tutorials/using-lcs-sierra-system):

Caveat: Do NOT use the MPIX_Query_cuda_support() routine or the preprocessor constant MPIX_CUDA_AWARE_SUPPORT to determine if Spectrum MPI is CUDA-aware. This routine has either been removed from the IBM implementation, or will always return false (older versions).

and this: https://github.com/lammps/lammps/pull/1582/files#diff-3335adca21d9025c15f1c0ba57812a56R43

simonbyrne commented 4 years ago

Also, what on earth is Open MPI v10.3.0.00rtm0? Their most recent version is 4.0.2