prepfold runs many processes regardless of -ncpus setting

[paulray]

With the new version of PRESTO, we are finding that on our Ubuntu linux machine, prepfold is spawning 36 processes to do a simple fold. For example, this runs at at CPU percentage of 3600

prepfold -o oink -par 0900-3144.par  guppi_55464_J1124-36_0002_subs_0001.fits

This is the same if you add -ncpus 2, or whatever.

Was there some change to the OpenMP code or something with the meson build that might cause this?

[scottransom]

Yikes! Yeah, I'm seeing the same thing. Bizarre. Can you tell me if you have OMP_NUM_THREADS set in your environment? I wonder if this is some library using threading by default?

[paulray]

I don't have any OMP environment variables set. In case it helps here are the linked libraries:

% ldd /Software/bin/prepfold 
    linux-vdso.so.1 (0x00007fff5e2ec000)
    libpresto.so => /Software/lib/x86_64-linux-gnu/libpresto.so (0x00007f952fee0000)
    libfftw3f.so.3 => /lib/x86_64-linux-gnu/libfftw3f.so.3 (0x00007f952fcb2000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f952fb63000)
    libcfitsio.so.8 => /lib/x86_64-linux-gnu/libcfitsio.so.8 (0x00007f952f864000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f952f841000)
    libcpgplot.so.5 => /lib/libcpgplot.so.5 (0x00007f952f836000)
    libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f952f56c000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f952f37a000)
    libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f952f338000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9530079000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f952f31c000)
    libcurl-gnutls.so.4 => /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007f952f28c000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f952f279000)
    libpgplot.so.5 => /lib/libpgplot.so.5 (0x00007f952f1eb000)
    libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f952f1a1000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f952f186000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f952f180000)
    libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f952f156000)
    libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f952f135000)
    librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f952f113000)
    libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007f952f0a4000)
    libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f952f091000)
    libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007f952f057000)
    libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f952ee81000)
    libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f952ee34000)
    libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f952eddc000)
    liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f952edcb000)
    libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007f952edbd000)
    libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f952ec80000)
    libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f952ec48000)
    libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f952eac4000)
    libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007f952ea8d000)
    libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f952ea09000)
    libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f952e732000)
    libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f952e5fc000)
    libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f952e5e6000)
    libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f952e507000)
    libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f952e4d6000)
    libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f952e4cf000)
    libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f952e4c0000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f952e4a4000)
    libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f952e487000)
    libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f952e440000)
    libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007f952e41d000)
    libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f952e3f3000)
    libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007f952e3e7000)
    libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f952e3e0000)
    libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f952e3d2000)
    libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f952e33f000)
    libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f952e299000)
    libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f952e261000)
    libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007f952e248000)
    libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f952e242000)
    libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f952e238000)
    libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007f952e20e000)
    libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f952e1fc000)
    libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f952e1ae000)
    libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f952e085000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f952e04a000)
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f952e02e000)

[scottransom]

So this is definitely due to including OpenMP in the build. You should be able to change this line in the top-level meson.build file: omp = dependency('openmp', required: false) to something like: omp = dependency('NO_openmp', required: false) so that meson can't find OpenMP, which is OK, since it isn't required. And the build will happen without it.

When I do that, I'm back to regular 100% CPU usage. I don't understand why this is happening, though, as I don't think it happened with the previous build with OpenMP! And you used to be able to turn OpenMP usage effectively off with the OMP_NUM_THREADS=1 environment variable.

I'll continue investigating.

[paulray]

The above solution did fix the issue.