MPICH building error: simple/lib: No such file or directory

[JiakunYan]

Version: the current master branch (7953471529c874df89cb08dd782431ac849d8b02) Platform: LSU Rostam (with Infiniband). Spack spec: mpich+vci+slurm~hydra netmod=ucx

Configure (copied from spack's build log):

Configuring MPICH version 4.3.0a1 with '--prefix=/home/jiakun/workspace/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/mpich-master-xyhqb6pqkytgpql3nf7p4bjqtoljpuv6' '--disable-silent-rules' '--enable-shared' '--with-pm=no' '--enable-romio' '--without-ibverbs' '--enable-wrapper-rpath=yes' '--with-yaksa=/home/jiakun/workspace/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/yaksa-0.2-62k6qthpwc7zusbgkwr6qv7bsxxlukjn' '--with-hwloc=/home/jiakun/workspace/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/hwloc-2.9.1-awinlsdglr5p7x5rfs5mw6hkjusqpeqz' '--with-slurm=yes' '--with-slurm-include=/usr/include' '--with-slurm-lib=/usr/lib' '--with-pmi=simple' '--without-cuda' '--without-hip' '--with-device=ch4:ucx' '--with-ucx=/home/jiakun/workspace/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/ucx-1.14.0-xez4qph5nrqwqdzgd36i454ine442i3z' '--enable-libxml2' '--enable-thread-cs=per-vci' '--with-datatype-engine=auto' 'CC=/home/jiakun/workspace/spack/lib/spack/env/gcc/gcc' 'CXX=/home/jiakun/workspace/spack/lib/spack/env/gcc/g++' 'FC=/home/jiakun/workspace/spack/lib/spack/env/gcc/gfortran' 'F77=/home/jiakun/workspace/spack/lib/spack/env/gcc/gfortran'

I got the following error message:

5 errors found in build log: 1701 libtool: compile: /home/jiakun/workspace/spack/lib/spack/env/gcc/gcc -DHAVE_CONFIG_H -I. -I./include -I./include -I/home/ji akun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/s pack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/ucx-1.14.0-xez4qph5nrqwqdzgd36i454ine442i3z/include -DNETMOD_INLINE=n etmod_inline_ucx -fvisibility=hidden -MT src/pmi_v2.lo -MD -MP -MF src/.deps/pmi_v2.Tpo -c src/pmi_v2.c -o src/pmi_v2.o >/ dev/null 2>&1 1702 libtool: compile: /home/jiakun/workspace/spack/lib/spack/env/gcc/gcc -DHAVE_CONFIG_H -I. -I./include -I./include -I/home/ji akun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/s pack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/ucx-1.14.0-xez4qph5nrqwqdzgd36i454ine442i3z/include -DNETMOD_INLINE=n etmod_inline_ucx -fvisibility=hidden -MT src/pmix.lo -MD -MP -MF src/.deps/pmix.Tpo -c src/pmix.c -o src/pmix.o >/dev/null 2>&1 1703 libtool: compile: /home/jiakun/workspace/spack/lib/spack/env/gcc/gcc -DHAVE_CONFIG_H -I. -I./include -I./include -I/home/ji akun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/s pack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/ucx-1.14.0-xez4qph5nrqwqdzgd36i454ine442i3z/include -DNETMOD_INLINE=n etmod_inline_ucx -fvisibility=hidden -MT src/pmi_v1.lo -MD -MP -MF src/.deps/pmi_v1.Tpo -c src/pmi_v1.c -o src/pmi_v1.o >/ dev/null 2>&1 1704 libtool: compile: /home/jiakun/workspace/spack/lib/spack/env/gcc/gcc -DHAVE_CONFIG_H -I. -I./include -I./include -I/home/ji akun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/s pack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/ucx-1.14.0-xez4qph5nrqwqdzgd36i454ine442i3z/include -DNETMOD_INLINE=n etmod_inline_ucx -fvisibility=hidden -MT src/pmi_msg.lo -MD -MP -MF src/.deps/pmi_msg.Tpo -c src/pmi_msg.c -o src/pmi_msg. o >/dev/null 2>&1 1705 libtool: compile: /home/jiakun/workspace/spack/lib/spack/env/gcc/gcc -DHAVE_CONFIG_H -I. -I./include -I./include -I/home/ji akun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/mpich-master/src/mpl/include -I/home/jiakun/workspace/s pack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/ucx-1.14.0-xez4qph5nrqwqdzgd36i454ine442i3z/include -DNETMOD_INLINE=n etmod_inline_ucx -fvisibility=hidden -MT src/pmi_wire.lo -MD -MP -MF src/.deps/pmi_wire.Tpo -c src/pmi_wire.c -o src/pmi_w ire.o >/dev/null 2>&1 1706 /bin/sh ./libtool --tag=CC --mode=link /home/jiakun/workspace/spack/lib/spack/env/gcc/gcc -fvisibility=hidden -L/home/j iakun/workspace/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/ucx-1.14.0-xez4qph5nrqwqdzgd36i454ine442i3z/lib -L/hom e/jiakun/workspace/spack/opt/spack/linux-rhel8-skylake_avx512/gcc-12.3.0/hwloc-2.9.1-awinlsdglr5p7x5rfs5mw6hkjusqpeqz/lib -L simple/lib -o libpmiutil.la src/pmi_wire.lo src/pmi_msg.lo src/pmi_common.lo src/pmi_util.lo

1707 ./libtool: line 7472: cd: simple/lib: No such file or directory 1708 libtool: error: cannot determine absolute directory name of 'simple/lib' 1709 make[3]: [Makefile:505: libpmiutil.la] Error 1 1710 make[3]: Leaving directory '/home/jiakun/workspace/mpich-master/src/pmi' 1711 make[2]: [Makefile:591: all-recursive] Error 1 1712 make[2]: Leaving directory '/home/jiakun/workspace/mpich-master/src/pmi' 1713 make[1]: [Makefile:29820: all-recursive] Error 1 1714 make[1]: Leaving directory '/home/jiakun/workspace/mpich-master' 1715 make: [Makefile:7597: all] Error 2

[JiakunYan]

After some investigation, I bypass this error with --with-pmi=slurm. However, there is no way to pass --with-pmi=slurm to MPICH configure through Spack so I have to manually change the MPICH Spack package.py.

Although it no longer bothers me, I am still not sure whether this "simple/pmi not found" is a MPICH bug or not.

[raffenet]

We'll update the Spack recipe to fix. Thanks for pointing it out.