Open paulray opened 1 month ago
Yikes! Yeah, I'm seeing the same thing. Bizarre. Can you tell me if you have OMP_NUM_THREADS set in your environment? I wonder if this is some library using threading by default?
I don't have any OMP environment variables set. In case it helps here are the linked libraries:
% ldd /Software/bin/prepfold
linux-vdso.so.1 (0x00007fff5e2ec000)
libpresto.so => /Software/lib/x86_64-linux-gnu/libpresto.so (0x00007f952fee0000)
libfftw3f.so.3 => /lib/x86_64-linux-gnu/libfftw3f.so.3 (0x00007f952fcb2000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f952fb63000)
libcfitsio.so.8 => /lib/x86_64-linux-gnu/libcfitsio.so.8 (0x00007f952f864000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f952f841000)
libcpgplot.so.5 => /lib/libcpgplot.so.5 (0x00007f952f836000)
libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f952f56c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f952f37a000)
libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f952f338000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9530079000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f952f31c000)
libcurl-gnutls.so.4 => /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007f952f28c000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f952f279000)
libpgplot.so.5 => /lib/libpgplot.so.5 (0x00007f952f1eb000)
libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f952f1a1000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f952f186000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f952f180000)
libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f952f156000)
libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f952f135000)
librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f952f113000)
libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007f952f0a4000)
libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f952f091000)
libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007f952f057000)
libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f952ee81000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f952ee34000)
libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f952eddc000)
liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f952edcb000)
libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007f952edbd000)
libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f952ec80000)
libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f952ec48000)
libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f952eac4000)
libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007f952ea8d000)
libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f952ea09000)
libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f952e732000)
libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f952e5fc000)
libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f952e5e6000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f952e507000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f952e4d6000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f952e4cf000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f952e4c0000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f952e4a4000)
libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f952e487000)
libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f952e440000)
libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007f952e41d000)
libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f952e3f3000)
libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007f952e3e7000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f952e3e0000)
libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f952e3d2000)
libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f952e33f000)
libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f952e299000)
libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f952e261000)
libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007f952e248000)
libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f952e242000)
libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f952e238000)
libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007f952e20e000)
libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f952e1fc000)
libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f952e1ae000)
libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f952e085000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f952e04a000)
libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f952e02e000)
So this is definitely due to including OpenMP in the build. You should be able to change this line in the top-level meson.build
file:
omp = dependency('openmp', required: false)
to something like:
omp = dependency('NO_openmp', required: false)
so that meson can't find OpenMP, which is OK, since it isn't required. And the build will happen without it.
When I do that, I'm back to regular 100% CPU usage. I don't understand why this is happening, though, as I don't think it happened with the previous build with OpenMP! And you used to be able to turn OpenMP usage effectively off with the OMP_NUM_THREADS=1
environment variable.
I'll continue investigating.
The above solution did fix the issue.
With the new version of PRESTO, we are finding that on our Ubuntu linux machine, prepfold is spawning 36 processes to do a simple fold. For example, this runs at at CPU percentage of 3600
This is the same if you add
-ncpus 2
, or whatever.Was there some change to the OpenMP code or something with the meson build that might cause this?