search

open-mpi / ompi

Open MPI main development repository
https://www.open-mpi.org
Other
2.17k stars 861 forks source link

Perl not detected (AC_MSG_ERROR) #12399

Closed hipersayanX closed 8 months ago

hipersayanX commented 8 months ago

When executing:

./autogen.pl --no-3rdparty 'openmix,pmix,libevent,hwloc,prrte'

I got this error:

configure.ac:87: error: possibly undefined macro: AC_MSG_ERROR
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.

Pointing to these lines

AC_CHECK_PROG([PERL],[perl],[perl],[no])
AS_IF([test "X$PERL" = "Xno"],
      [AC_MSG_ERROR(["Open MPI requires perl. Aborting"])])

But Perl is actually installed:

perl -v
This is perl 5, version 38, subversion 2 (v5.38.2) built for x86_64-linux-thread-multi

How can I fix it?

ggouaillardet commented 8 months ago

I think the error is related to the AC_MSG_ERROR macro and not to its context/argument (e.g. perl)

What is your environment and which versions of libtool, autoconf and automake are you using?

hipersayanX commented 8 months ago 
uname -a

Linux localhost.localdomain 6.6.21-1-lts #1 SMP PREEMPT_DYNAMIC Wed, 06 Mar 2024 16:59:55 +0000 x86_64 GNU/Linux
cat /etc/*release*

DISTRIB_ID="Arch"
DISTRIB_RELEASE="rolling"
DISTRIB_DESCRIPTION="Arch Linux"
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo
libtool (GNU libtool) 2.4.7.4-1ec8f-dirty
autoconf (GNU Autoconf) 2.72
automake (GNU automake) 1.16.5

I'm trying to cross compile to Android, I have already compiled openmix, libevent, hwloc, and prrte. I was taking the sources from https://www.open-mpi.org/software/ompi/ (and it was compiling fine with many patches), but the automated builds does not seems to be taking the URL, so I have decided to take the sources from here.

jsquyres commented 8 months ago

I have seen this kind of thing happen when Autoconf, Automake, and Libtool are not installed into the same $prefix.

Are you building / installing AC, AM, and LT by hand, perchance? Or are these distro-installed AC, AM, and LT packages?

From: hipersayanX @.> Sent: Monday, March 11, 2024 10:54 AM To: open-mpi/ompi @.> Cc: Subscribed @.***> Subject: Re: [open-mpi/ompi] Perl not detected (AC_MSG_ERROR) (Issue #12399)

uname -a

Linux localhost.localdomain 6.6.21-1-lts #1 SMP PREEMPT_DYNAMIC Wed, 06 Mar 2024 16:59:55 +0000 x86_64 GNU/Linux

cat /etc/release

DISTRIB_ID="Arch" DISTRIB_RELEASE="rolling" DISTRIB_DESCRIPTION="Arch Linux" NAME="Arch Linux" PRETTY_NAME="Arch Linux" ID=arch BUILD_ID=rolling ANSI_COLOR="38;2;23;147;209" HOME_URL="https://archlinux.org/" DOCUMENTATION_URL="https://wiki.archlinux.org/" SUPPORT_URL="https://bbs.archlinux.org/" BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues" PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/" LOGO=archlinux-logo

libtool (GNU libtool) 2.4.7.4-1ec8f-dirty autoconf (GNU Autoconf) 2.72 automake (GNU automake) 1.16.5

I'm trying to cross compile to Android, I have already compiled openmix, libevent, hwloc, and prrte. I was taking the sources from https://www.open-mpi.org/software/ompi/ (and it was compiling fine with many patches), but the automated buildshttps://github.com/Martchus/PKGBUILDs/issues/175#issuecomment-1988005963 does not seems to be taking the URL, so I have decided to take the sources from here.

— Reply to this email directly, view it on GitHubhttps://github.com/open-mpi/ompi/issues/12399#issuecomment-1988634103, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAEWWZERLRCYNEQHI5L263LYXXASTAVCNFSM6AAAAABEQNE2O2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBYGYZTIMJQGM. You are receiving this because you are subscribed to this thread.Message ID: @.***>

hipersayanX commented 8 months ago

@jsquyres

Or are these distro-installed AC, AM, and LT packages?

Yes, distro installed.

ggouaillardet commented 8 months ago

I will have a look. Any reason why you are not building from a tarball (so you would not need to run autogen.pl)?

hipersayanX commented 8 months ago

AC_MSG_WARN and AC_MSG_CHECKING also failing, right now I'm try commenting the conflictive code just for the purpose of bypassing the build, but this is not the right way to do the things for sure.

@ggouaillardet

As explained before:

I'm trying to cross compile to Android, I have already compiled openmix, libevent, hwloc, and prrte. I was taking the sources from https://www.open-mpi.org/software/ompi/ (and it was compiling fine with many patches), but the automated builds does not seems to be taking the URL, so I have decided to take the sources from here.

This was the build script that I was using, and it was compiling more or less fine (applying many patches), but the URL seems to be failing on @Martchus side, see android-aarch64-openmpi log.

hipersayanX commented 8 months ago

Where are those macros defined?

ggouaillardet commented 8 months ago

IIRC I faced a similar issue and the root cause was an undefined macro that has no obvious link with the error message (!). I ended up doing a bisection like thing by removing some content from configure.ac until I found the offending line.

hipersayanX commented 8 months ago

@ggouaillardet the problem is that the code in configure.ac is a tangle, and I barely understand what's going on there :laughing: I tried greping the missing macros, but it does not seems to be defined inside the project.

jsquyres commented 8 months ago

Those macros are defined in Autoconf. When you get error messages like this, it is almost always due to a faulty installation of AC, AM, and/or LT (or they are not integrated properly into each other, such as being installed into different $prefixes, and/or some other way of not having their data / plugin directories not shared with each other properly).

From: hipersayanX @.> Sent: Monday, March 11, 2024 11:57 AM To: open-mpi/ompi @.> Cc: Jeff Squyres (jsquyres) @.>; Mention @.> Subject: Re: [open-mpi/ompi] Perl not detected (AC_MSG_ERROR) (Issue #12399)

@ggouaillardethttps://github.com/ggouaillardet the problem is that the code in configure.ac is a tangle, and I barely understand what's going on there 😆 I tried greping the missing macros, but it does not seems to be defined inside the project.

— Reply to this email directly, view it on GitHubhttps://github.com/open-mpi/ompi/issues/12399#issuecomment-1988783836, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAEWWZEJKSYMN4RXZUZTTLDYXXIAHAVCNFSM6AAAAABEQNE2O2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBYG44DGOBTGY. You are receiving this because you were mentioned.Message ID: @.***>

hipersayanX commented 8 months ago

@jsquyres

these are the installed files by each package:

pacman -Ql autoconf

autoconf /usr/
autoconf /usr/bin/
autoconf /usr/bin/autoconf
autoconf /usr/bin/autoheader
autoconf /usr/bin/autom4te
autoconf /usr/bin/autoreconf
autoconf /usr/bin/autoscan
autoconf /usr/bin/autoupdate
autoconf /usr/bin/ifnames
autoconf /usr/share/
autoconf /usr/share/autoconf/
autoconf /usr/share/autoconf/Autom4te/
autoconf /usr/share/autoconf/Autom4te/C4che.pm
autoconf /usr/share/autoconf/Autom4te/ChannelDefs.pm
autoconf /usr/share/autoconf/Autom4te/Channels.pm
autoconf /usr/share/autoconf/Autom4te/Config.pm
autoconf /usr/share/autoconf/Autom4te/Configure_ac.pm
autoconf /usr/share/autoconf/Autom4te/FileUtils.pm
autoconf /usr/share/autoconf/Autom4te/General.pm
autoconf /usr/share/autoconf/Autom4te/Getopt.pm
autoconf /usr/share/autoconf/Autom4te/Request.pm
autoconf /usr/share/autoconf/Autom4te/XFile.pm
autoconf /usr/share/autoconf/INSTALL
autoconf /usr/share/autoconf/autoconf/
autoconf /usr/share/autoconf/autoconf/autoconf.m4
autoconf /usr/share/autoconf/autoconf/autoconf.m4f
autoconf /usr/share/autoconf/autoconf/autoheader.m4
autoconf /usr/share/autoconf/autoconf/autoscan.m4
autoconf /usr/share/autoconf/autoconf/autotest.m4
autoconf /usr/share/autoconf/autoconf/autoupdate.m4
autoconf /usr/share/autoconf/autoconf/c.m4
autoconf /usr/share/autoconf/autoconf/erlang.m4
autoconf /usr/share/autoconf/autoconf/fortran.m4
autoconf /usr/share/autoconf/autoconf/functions.m4
autoconf /usr/share/autoconf/autoconf/general.m4
autoconf /usr/share/autoconf/autoconf/go.m4
autoconf /usr/share/autoconf/autoconf/headers.m4
autoconf /usr/share/autoconf/autoconf/lang.m4
autoconf /usr/share/autoconf/autoconf/libs.m4
autoconf /usr/share/autoconf/autoconf/oldnames.m4
autoconf /usr/share/autoconf/autoconf/programs.m4
autoconf /usr/share/autoconf/autoconf/specific.m4
autoconf /usr/share/autoconf/autoconf/status.m4
autoconf /usr/share/autoconf/autoconf/trailer.m4
autoconf /usr/share/autoconf/autoconf/types.m4
autoconf /usr/share/autoconf/autom4te.cfg
autoconf /usr/share/autoconf/autoscan/
autoconf /usr/share/autoconf/autoscan/autoscan.list
autoconf /usr/share/autoconf/autotest/
autoconf /usr/share/autoconf/autotest/autotest.m4
autoconf /usr/share/autoconf/autotest/autotest.m4f
autoconf /usr/share/autoconf/autotest/general.m4
autoconf /usr/share/autoconf/autotest/specific.m4
autoconf /usr/share/autoconf/build-aux/
autoconf /usr/share/autoconf/build-aux/config.guess
autoconf /usr/share/autoconf/build-aux/config.sub
autoconf /usr/share/autoconf/build-aux/install-sh
autoconf /usr/share/autoconf/m4sugar/
autoconf /usr/share/autoconf/m4sugar/foreach.m4
autoconf /usr/share/autoconf/m4sugar/m4sh.m4
autoconf /usr/share/autoconf/m4sugar/m4sh.m4f
autoconf /usr/share/autoconf/m4sugar/m4sugar.m4
autoconf /usr/share/autoconf/m4sugar/m4sugar.m4f
autoconf /usr/share/autoconf/version.m4
autoconf /usr/share/info/
autoconf /usr/share/info/autoconf.info.gz
autoconf /usr/share/licenses/
autoconf /usr/share/licenses/autoconf/
autoconf /usr/share/licenses/autoconf/COPYING.EXCEPTION
autoconf /usr/share/man/
autoconf /usr/share/man/man1/
autoconf /usr/share/man/man1/autoconf.1.gz
autoconf /usr/share/man/man1/autoheader.1.gz
autoconf /usr/share/man/man1/autom4te.1.gz
autoconf /usr/share/man/man1/autoreconf.1.gz
autoconf /usr/share/man/man1/autoscan.1.gz
autoconf /usr/share/man/man1/autoupdate.1.gz
autoconf /usr/share/man/man1/ifnames.1.gz

pacman -Ql automake

automake /usr/
automake /usr/bin/
automake /usr/bin/aclocal
automake /usr/bin/aclocal-1.16
automake /usr/bin/automake
automake /usr/bin/automake-1.16
automake /usr/share/
automake /usr/share/aclocal-1.16/
automake /usr/share/aclocal-1.16/amversion.m4
automake /usr/share/aclocal-1.16/ar-lib.m4
automake /usr/share/aclocal-1.16/as.m4
automake /usr/share/aclocal-1.16/auxdir.m4
automake /usr/share/aclocal-1.16/cond-if.m4
automake /usr/share/aclocal-1.16/cond.m4
automake /usr/share/aclocal-1.16/depend.m4
automake /usr/share/aclocal-1.16/depout.m4
automake /usr/share/aclocal-1.16/dmalloc.m4
automake /usr/share/aclocal-1.16/extra-recurs.m4
automake /usr/share/aclocal-1.16/gcj.m4
automake /usr/share/aclocal-1.16/init.m4
automake /usr/share/aclocal-1.16/install-sh.m4
automake /usr/share/aclocal-1.16/internal/
automake /usr/share/aclocal-1.16/internal/ac-config-macro-dirs.m4
automake /usr/share/aclocal-1.16/lead-dot.m4
automake /usr/share/aclocal-1.16/lex.m4
automake /usr/share/aclocal-1.16/lispdir.m4
automake /usr/share/aclocal-1.16/maintainer.m4
automake /usr/share/aclocal-1.16/make.m4
automake /usr/share/aclocal-1.16/missing.m4
automake /usr/share/aclocal-1.16/mkdirp.m4
automake /usr/share/aclocal-1.16/obsolete.m4
automake /usr/share/aclocal-1.16/options.m4
automake /usr/share/aclocal-1.16/prog-cc-c-o.m4
automake /usr/share/aclocal-1.16/python.m4
automake /usr/share/aclocal-1.16/runlog.m4
automake /usr/share/aclocal-1.16/sanity.m4
automake /usr/share/aclocal-1.16/silent.m4
automake /usr/share/aclocal-1.16/strip.m4
automake /usr/share/aclocal-1.16/substnot.m4
automake /usr/share/aclocal-1.16/tar.m4
automake /usr/share/aclocal-1.16/upc.m4
automake /usr/share/aclocal-1.16/vala.m4
automake /usr/share/aclocal/
automake /usr/share/aclocal/README
automake /usr/share/automake-1.16/
automake /usr/share/automake-1.16/Automake/
automake /usr/share/automake-1.16/Automake/ChannelDefs.pm
automake /usr/share/automake-1.16/Automake/Channels.pm
automake /usr/share/automake-1.16/Automake/Condition.pm
automake /usr/share/automake-1.16/Automake/Config.pm
automake /usr/share/automake-1.16/Automake/Configure_ac.pm
automake /usr/share/automake-1.16/Automake/DisjConditions.pm
automake /usr/share/automake-1.16/Automake/FileUtils.pm
automake /usr/share/automake-1.16/Automake/General.pm
automake /usr/share/automake-1.16/Automake/Getopt.pm
automake /usr/share/automake-1.16/Automake/Item.pm
automake /usr/share/automake-1.16/Automake/ItemDef.pm
automake /usr/share/automake-1.16/Automake/Language.pm
automake /usr/share/automake-1.16/Automake/Location.pm
automake /usr/share/automake-1.16/Automake/Options.pm
automake /usr/share/automake-1.16/Automake/Rule.pm
automake /usr/share/automake-1.16/Automake/RuleDef.pm
automake /usr/share/automake-1.16/Automake/VarDef.pm
automake /usr/share/automake-1.16/Automake/Variable.pm
automake /usr/share/automake-1.16/Automake/Version.pm
automake /usr/share/automake-1.16/Automake/Wrap.pm
automake /usr/share/automake-1.16/Automake/XFile.pm
automake /usr/share/automake-1.16/COPYING
automake /usr/share/automake-1.16/INSTALL
automake /usr/share/automake-1.16/am/
automake /usr/share/automake-1.16/am/check.am
automake /usr/share/automake-1.16/am/check2.am
automake /usr/share/automake-1.16/am/clean-hdr.am
automake /usr/share/automake-1.16/am/clean.am
automake /usr/share/automake-1.16/am/compile.am
automake /usr/share/automake-1.16/am/configure.am
automake /usr/share/automake-1.16/am/data.am
automake /usr/share/automake-1.16/am/dejagnu.am
automake /usr/share/automake-1.16/am/depend.am
automake /usr/share/automake-1.16/am/depend2.am
automake /usr/share/automake-1.16/am/distdir.am
automake /usr/share/automake-1.16/am/footer.am
automake /usr/share/automake-1.16/am/header-vars.am
automake /usr/share/automake-1.16/am/header.am
automake /usr/share/automake-1.16/am/inst-vars.am
automake /usr/share/automake-1.16/am/install.am
automake /usr/share/automake-1.16/am/java.am
automake /usr/share/automake-1.16/am/lang-compile.am
automake /usr/share/automake-1.16/am/lex.am
automake /usr/share/automake-1.16/am/library.am
automake /usr/share/automake-1.16/am/libs.am
automake /usr/share/automake-1.16/am/libtool.am
automake /usr/share/automake-1.16/am/lisp.am
automake /usr/share/automake-1.16/am/ltlib.am
automake /usr/share/automake-1.16/am/ltlibrary.am
automake /usr/share/automake-1.16/am/mans-vars.am
automake /usr/share/automake-1.16/am/mans.am
automake /usr/share/automake-1.16/am/program.am
automake /usr/share/automake-1.16/am/progs.am
automake /usr/share/automake-1.16/am/python.am
automake /usr/share/automake-1.16/am/remake-hdr.am
automake /usr/share/automake-1.16/am/scripts.am
automake /usr/share/automake-1.16/am/subdirs.am
automake /usr/share/automake-1.16/am/tags.am
automake /usr/share/automake-1.16/am/texi-vers.am
automake /usr/share/automake-1.16/am/texibuild.am
automake /usr/share/automake-1.16/am/texinfos.am
automake /usr/share/automake-1.16/am/vala.am
automake /usr/share/automake-1.16/am/yacc.am
automake /usr/share/automake-1.16/ar-lib
automake /usr/share/automake-1.16/compile
automake /usr/share/automake-1.16/config.guess
automake /usr/share/automake-1.16/config.sub
automake /usr/share/automake-1.16/depcomp
automake /usr/share/automake-1.16/install-sh
automake /usr/share/automake-1.16/mdate-sh
automake /usr/share/automake-1.16/missing
automake /usr/share/automake-1.16/mkinstalldirs
automake /usr/share/automake-1.16/py-compile
automake /usr/share/automake-1.16/tap-driver.sh
automake /usr/share/automake-1.16/test-driver
automake /usr/share/automake-1.16/texinfo.tex
automake /usr/share/automake-1.16/ylwrap
automake /usr/share/doc/
automake /usr/share/doc/automake/
automake /usr/share/doc/automake/amhello-1.0.tar.gz
automake /usr/share/info/
automake /usr/share/info/automake-history.info.gz
automake /usr/share/info/automake.info-1.gz
automake /usr/share/info/automake.info-2.gz
automake /usr/share/info/automake.info.gz
automake /usr/share/man/
automake /usr/share/man/man1/
automake /usr/share/man/man1/aclocal-1.16.1.gz
automake /usr/share/man/man1/aclocal.1.gz
automake /usr/share/man/man1/automake-1.16.1.gz
automake /usr/share/man/man1/automake.1.gz

pacman -Ql libtool

libtool /usr/
libtool /usr/bin/
libtool /usr/bin/libtool
libtool /usr/bin/libtoolize
libtool /usr/include/
libtool /usr/include/libltdl/
libtool /usr/include/libltdl/lt_dlloader.h
libtool /usr/include/libltdl/lt_error.h
libtool /usr/include/libltdl/lt_system.h
libtool /usr/include/ltdl.h
libtool /usr/lib/
libtool /usr/lib/libltdl.so
libtool /usr/lib/libltdl.so.7
libtool /usr/lib/libltdl.so.7.3.2
libtool /usr/share/
libtool /usr/share/aclocal/
libtool /usr/share/aclocal/libtool.m4
libtool /usr/share/aclocal/ltargz.m4
libtool /usr/share/aclocal/ltdl.m4
libtool /usr/share/aclocal/ltoptions.m4
libtool /usr/share/aclocal/ltsugar.m4
libtool /usr/share/aclocal/ltversion.m4
libtool /usr/share/aclocal/lt~obsolete.m4
libtool /usr/share/info/
libtool /usr/share/info/libtool.info-1.gz
libtool /usr/share/info/libtool.info-2.gz
libtool /usr/share/info/libtool.info.gz
libtool /usr/share/libtool/
libtool /usr/share/libtool/COPYING.LIB
libtool /usr/share/libtool/Makefile.am
libtool /usr/share/libtool/Makefile.in
libtool /usr/share/libtool/README
libtool /usr/share/libtool/aclocal.m4
libtool /usr/share/libtool/build-aux/
libtool /usr/share/libtool/build-aux/compile
libtool /usr/share/libtool/build-aux/config.guess
libtool /usr/share/libtool/build-aux/config.sub
libtool /usr/share/libtool/build-aux/depcomp
libtool /usr/share/libtool/build-aux/install-sh
libtool /usr/share/libtool/build-aux/ltmain.sh
libtool /usr/share/libtool/build-aux/missing
libtool /usr/share/libtool/config-h.in
libtool /usr/share/libtool/configure
libtool /usr/share/libtool/configure.ac
libtool /usr/share/libtool/libltdl/
libtool /usr/share/libtool/libltdl/lt__alloc.h
libtool /usr/share/libtool/libltdl/lt__argz_.h
libtool /usr/share/libtool/libltdl/lt__dirent.h
libtool /usr/share/libtool/libltdl/lt__glibc.h
libtool /usr/share/libtool/libltdl/lt__private.h
libtool /usr/share/libtool/libltdl/lt__strl.h
libtool /usr/share/libtool/libltdl/lt_dlloader.h
libtool /usr/share/libtool/libltdl/lt_error.h
libtool /usr/share/libtool/libltdl/lt_system.h
libtool /usr/share/libtool/libltdl/slist.h
libtool /usr/share/libtool/loaders/
libtool /usr/share/libtool/loaders/dld_link.c
libtool /usr/share/libtool/loaders/dlopen.c
libtool /usr/share/libtool/loaders/dyld.c
libtool /usr/share/libtool/loaders/load_add_on.c
libtool /usr/share/libtool/loaders/loadlibrary.c
libtool /usr/share/libtool/loaders/preopen.c
libtool /usr/share/libtool/loaders/shl_load.c
libtool /usr/share/libtool/lt__alloc.c
libtool /usr/share/libtool/lt__argz.c
libtool /usr/share/libtool/lt__dirent.c
libtool /usr/share/libtool/lt__strl.c
libtool /usr/share/libtool/lt_dlloader.c
libtool /usr/share/libtool/lt_error.c
libtool /usr/share/libtool/ltdl.c
libtool /usr/share/libtool/ltdl.h
libtool /usr/share/libtool/ltdl.mk
libtool /usr/share/libtool/slist.c
libtool /usr/share/man/
libtool /usr/share/man/man1/
libtool /usr/share/man/man1/libtool.1.gz
libtool /usr/share/man/man1/libtoolize.1.gz
jsquyres commented 8 months ago

You should probably test out a trivial AC/AM/LT package on your system (i.e., something significantly simpler than Open MPI), and make sure that AC/AM/LT are working.

Additionally, can you send the full output from autogen.pl? Sometimes the real error message gets hidden much further up in the output (i.e., not at the very end), and it causes cascading failures that ultimately ends up in AC/AM/LT failing.

hipersayanX commented 8 months ago

Here is the full log:

https://gist.github.com/hipersayanX/8303ecc94451cb8fd931b433737c7ec6

jsquyres commented 8 months ago

Thanks for the full log. It didn't appear to show any additional errors.

  1. Can you supply all the other information that was requested in the github issue template?

    https://github.com/open-mpi/ompi/blob/main/.github/ISSUE_TEMPLATE/bug_report.md

  2. Can you confirm if a simpler package that uses AC/AM/LT works properly on your system?

hipersayanX commented 8 months ago

@jsquyres

Minimal example and output

https://gist.github.com/hipersayanX/6865095414bd7add4c5bc7d03bb8c0cf

It seems to be working fine.

Also:

  • Computer hardware:
  • Network type:

Is this really necessary for this? because I don't think it much related to this the problem.

jsquyres commented 8 months ago

Can you add Libtool into that minimal example? AC+AM is not quite the same thing as AC+AM+LT.

I'm not saying that Open MPI isn't at fault, but it would be surprising since this isn't failing anywhere else. Hence, more detail from you would be good. E.g., data that is related to this problem is the version of Open MPI that you're using, how you're obtaining it, the git hashes if you're obtaining from git, ... etc. Please provide all relevant information, otherwise, it's difficult to help you.

hipersayanX commented 8 months ago

Can you add Libtool into that minimal example? AC+AM is not quite the same thing as AC+AM+LT.

I took the example from here, I don't have experience using AC+AM+LT, so I can't provide a better test because I don't know how to use it.

the version of Open MPI that you're using, how you're obtaining it, the git hashes if you're obtaining from git, ... etc.

I'm downloading this one:

https://github.com/open-mpi/ompi/releases/tag/v5.0.2 https://github.com/open-mpi/ompi/archive/refs/tags/v5.0.2.tar.gz

ggouaillardet commented 8 months ago

FWIW, ./autogen.pl --force --no-3rdparty openpmix,libevent,hwloc,prrte worked just fine on my (mostly) up-to-date archlinux box.

you should not use the tarballs from GitHub but instead download the official ones from www.open-mpi.org. also, you will need to use the --force flag when running autogen.pl on an official tarball.

hipersayanX commented 8 months ago

@ggouaillardet

Tried, same result

you should not use the tarballs from GitHub but instead download the official ones from www.open-mpi.org. also, you will need to use the --force flag when running autogen.pl on an official tarball.

Yeah, at the end I have reverted back the script, but still think its weird why its not working in my system.

Also tried executing in bash (I was using zsh), same result. Also tried executing the failing command

autoconf --include=config --include=config/oac --force

same result, I don't know what could be the problem :confused:

ggouaillardet commented 8 months ago

Here is a simple singularity file that just works

Bootstrap: docker
From: archlinux

%post
set -x
pacman -Sy --noconfirm wget
wget https://www.open-mpi.org/software/ompi/v5.0/downloads/openmpi-5.0.2.tar.bz2
tar xvfj openmpi-5.0.2.tar.bz2
cd openmpi-5.0.2
pacman -Sy --noconfirm perl autoconf automake libtool
./autogen.pl --force --no-3rdparty openpmix,libevent,hwloc,prrte

and then

$ singularity build --fakeroot ompi.sif ompi.def

That strongly suggests the issue is specific to your environment.

jsquyres commented 8 months ago

@hipersayanX Try this example:

#!/bin/bash                                                                                  

set -euxo pipefail

rm -rf example-1
mkdir -p example-1
cd example-1

cat << EOF > configure.ac                                                                    
AC_INIT([hello], [1.0])                                                                      
AC_CONFIG_SRCDIR([hello.c])                                                                  
AC_CONFIG_AUX_DIR([build-aux])                                                               
AM_INIT_AUTOMAKE([-Wall -Werror foreign])                                                    
AC_PROG_CC                                                                                   
AM_PROG_AR                                                                                   
LT_INIT                                                                                      
AC_CONFIG_FILES([Makefile])                                                                  
AC_OUTPUT                                                                                    
EOF                                                                                          

cat << EOF > Makefile.am                                                                     
bin_PROGRAMS = hello                                                                         
hello_SOURCES = main.c                                                                       
hello_LDADD = libhello.la                                                                    

lib_LTLIBRARIES = libhello.la                                                                
libhello_la_SOURCES = hello.c                                                                
EOF                                                                                          

cat << EOF > main.c                                                                          
void hello(void);                                                                            
int main(int argc, char** argv) {                                                            
  hello();                                                                                   
  return 0;                                                                                  
}                                                                                            
EOF                                                                                          

cat <<EOF > hello.c                                                                          
#include <stdio.h>                                                                           
void hello(void) {                                                                           
  printf("Hello, world!\n");                                                                 
}                                                                                            
EOF                                                                                          

autoreconf --verbose --install --force
./configure
make
./hello
make distcheck
hipersayanX commented 8 months ago

@ggouaillardet not the same source, try with:

wget https://github.com/open-mpi/ompi/archive/refs/tags/v5.0.2.tar.gz
hipersayanX commented 8 months ago

@jsquyres

It seems to be working fine.

+ rm -rf example-1
+ mkdir -p example-1
+ cd example-1
+ cat
+ cat
+ cat
+ cat
+ autoreconf --verbose --install --force
autoreconf: export WARNINGS=
autoreconf: Entering directory '.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force 
autoreconf: configure.ac: tracing
autoreconf: configure.ac: creating directory build-aux
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: Consider adding 'AC_CONFIG_MACRO_DIRS([m4])' to configure.ac,
libtoolize: and rerunning libtoolize and aclocal.
libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
autoreconf: configure.ac: not using Intltool
autoreconf: configure.ac: not using Gtkdoc
autoreconf: running: aclocal --force 
autoreconf: running: /usr/bin/autoconf --force
autoreconf: configure.ac: not using Autoheader
autoreconf: running: automake --add-missing --copy --force-missing
configure.ac:6: installing 'build-aux/ar-lib'
configure.ac:5: installing 'build-aux/compile'
configure.ac:7: installing 'build-aux/config.guess'
configure.ac:7: installing 'build-aux/config.sub'
configure.ac:4: installing 'build-aux/install-sh'
configure.ac:4: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
autoreconf: 'build-aux/install-sh' is updated
autoreconf: 'build-aux/config.sub' is updated
autoreconf: 'build-aux/config.guess' is updated
autoreconf: Leaving directory '.'
+ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for file... file
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands
+ make
gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"hello\ 1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I.     -g -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c
mv -f .deps/main.Tpo .deps/main.Po
/bin/sh ./libtool  --tag=CC   --mode=compile gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"hello\ 1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I.     -g -O2 -MT hello.lo -MD -MP -MF .deps/hello.Tpo -c -o hello.lo hello.c
libtool: compile:  gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" "-DPACKAGE_STRING=\"hello 1.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I. -g -O2 -MT hello.lo -MD -MP -MF .deps/hello.Tpo -c hello.c  -fPIC -DPIC -o .libs/hello.o
libtool: compile:  gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" "-DPACKAGE_STRING=\"hello 1.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I. -g -O2 -MT hello.lo -MD -MP -MF .deps/hello.Tpo -c hello.c -o hello.o >/dev/null 2>&1
mv -f .deps/hello.Tpo .deps/hello.Plo
/bin/sh ./libtool  --tag=CC   --mode=link gcc  -g -O2   -o libhello.la -rpath /usr/local/lib hello.lo  
libtool: link: gcc -shared  -fPIC -DPIC  .libs/hello.o    -g -O2   -Wl,-soname -Wl,libhello.so.0 -o .libs/libhello.so.0.0.0
libtool: link: (cd ".libs" && rm -f "libhello.so.0" && ln -s "libhello.so.0.0.0" "libhello.so.0")
libtool: link: (cd ".libs" && rm -f "libhello.so" && ln -s "libhello.so.0.0.0" "libhello.so")
libtool: link: ar cr .libs/libhello.a  hello.o
libtool: link: ranlib .libs/libhello.a
libtool: link: ( cd ".libs" && rm -f "libhello.la" && ln -s "../libhello.la" "libhello.la" )
/bin/sh ./libtool  --tag=CC   --mode=link gcc  -g -O2   -o hello main.o libhello.la 
libtool: link: gcc -g -O2 -o .libs/hello main.o  ./.libs/libhello.so -Wl,-rpath -Wl,/usr/local/lib
+ ./hello
Hello, world!
+ make distcheck
make  dist-gzip am__post_remove_distdir='@:'
make[1]: Entering directory '/home/user/actest/example-1'
make  distdir-am
make[2]: Entering directory '/home/user/actest/example-1'
if test -d "hello-1.0"; then find "hello-1.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "hello-1.0" || { sleep 5 && rm -rf "hello-1.0"; }; else :; fi
test -d "hello-1.0" || mkdir "hello-1.0"
test -n "" \
|| find "hello-1.0" -type d ! -perm -755 \
        -exec chmod u+rwx,go+rx {} \; -o \
  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
  ! -type d ! -perm -444 -exec /bin/sh /home/user/actest/example-1/build-aux/install-sh -c -m a+r {} {} \; \
|| chmod -R a+r "hello-1.0"
make[2]: Leaving directory '/home/user/actest/example-1'
tardir=hello-1.0 && ${TAR-tar} chof - "$tardir" | eval GZIP= gzip --best -c >hello-1.0.tar.gz
make[1]: Leaving directory '/home/user/actest/example-1'
if test -d "hello-1.0"; then find "hello-1.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "hello-1.0" || { sleep 5 && rm -rf "hello-1.0"; }; else :; fi
case 'hello-1.0.tar.gz' in \
*.tar.gz*) \
  eval GZIP= gzip --best -dc hello-1.0.tar.gz | ${TAR-tar} xf - ;;\
*.tar.bz2*) \
  bzip2 -dc hello-1.0.tar.bz2 | ${TAR-tar} xf - ;;\
*.tar.lz*) \
  lzip -dc hello-1.0.tar.lz | ${TAR-tar} xf - ;;\
*.tar.xz*) \
  xz -dc hello-1.0.tar.xz | ${TAR-tar} xf - ;;\
*.tar.Z*) \
  uncompress -c hello-1.0.tar.Z | ${TAR-tar} xf - ;;\
*.shar.gz*) \
  eval GZIP= gzip --best -dc hello-1.0.shar.gz | unshar ;;\
*.zip*) \
  unzip hello-1.0.zip ;;\
*.tar.zst*) \
  zstd -dc hello-1.0.tar.zst | ${TAR-tar} xf - ;;\
esac
chmod -R a-w hello-1.0
chmod u+w hello-1.0
mkdir hello-1.0/_build hello-1.0/_build/sub hello-1.0/_inst
chmod a-w hello-1.0
test -d hello-1.0/_build || exit 0; \
dc_install_base=`CDPATH="${ZSH_VERSION+.}:" && cd hello-1.0/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \
  && dc_destdir="${TMPDIR-/tmp}/am-dc-$$/" \
  && am__cwd=`pwd` \
  && CDPATH="${ZSH_VERSION+.}:" && cd hello-1.0/_build/sub \
  && ../../configure \
     \
     \
    --srcdir=../.. --prefix="$dc_install_base" \
  && make  \
  && make  dvi \
  && make  check \
  && make  install \
  && make  installcheck \
  && make  uninstall \
  && make  distuninstallcheck_dir="$dc_install_base" \
        distuninstallcheck \
  && chmod -R a-w "$dc_install_base" \
  && ({ \
       (cd ../.. && umask 077 && mkdir "$dc_destdir") \
       && make  DESTDIR="$dc_destdir" install \
       && make  DESTDIR="$dc_destdir" uninstall \
       && make  DESTDIR="$dc_destdir" \
            distuninstallcheck_dir="$dc_destdir" distuninstallcheck; \
      } || { rm -rf "$dc_destdir"; exit 1; }) \
  && rm -rf "$dc_destdir" \
  && make  dist \
  && rm -rf hello-1.0.tar.gz \
  && make  distcleancheck \
  && cd "$am__cwd" \
  || exit 1
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for file... file
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"hello\ 1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I. -I../..     -g -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o ../../main.c
mv -f .deps/main.Tpo .deps/main.Po
/bin/sh ./libtool  --tag=CC   --mode=compile gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"hello\ 1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I. -I../..     -g -O2 -MT hello.lo -MD -MP -MF .deps/hello.Tpo -c -o hello.lo ../../hello.c
libtool: compile:  gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" "-DPACKAGE_STRING=\"hello 1.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I. -I../.. -g -O2 -MT hello.lo -MD -MP -MF .deps/hello.Tpo -c ../../hello.c  -fPIC -DPIC -o .libs/hello.o
libtool: compile:  gcc -DPACKAGE_NAME=\"hello\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" "-DPACKAGE_STRING=\"hello 1.0\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -I. -I../.. -g -O2 -MT hello.lo -MD -MP -MF .deps/hello.Tpo -c ../../hello.c -o hello.o >/dev/null 2>&1
mv -f .deps/hello.Tpo .deps/hello.Plo
/bin/sh ./libtool  --tag=CC   --mode=link gcc  -g -O2   -o libhello.la -rpath /home/user/actest/example-1/hello-1.0/_inst/lib hello.lo  
libtool: link: gcc -shared  -fPIC -DPIC  .libs/hello.o    -g -O2   -Wl,-soname -Wl,libhello.so.0 -o .libs/libhello.so.0.0.0
libtool: link: (cd ".libs" && rm -f "libhello.so.0" && ln -s "libhello.so.0.0.0" "libhello.so.0")
libtool: link: (cd ".libs" && rm -f "libhello.so" && ln -s "libhello.so.0.0.0" "libhello.so")
libtool: link: ar cr .libs/libhello.a  hello.o
libtool: link: ranlib .libs/libhello.a
libtool: link: ( cd ".libs" && rm -f "libhello.la" && ln -s "../libhello.la" "libhello.la" )
/bin/sh ./libtool  --tag=CC   --mode=link gcc  -g -O2   -o hello main.o libhello.la 
libtool: link: gcc -g -O2 -o .libs/hello main.o  ./.libs/libhello.so -Wl,-rpath -Wl,/home/user/actest/example-1/hello-1.0/_inst/lib
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Nothing to be done for 'dvi'.
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Nothing to be done for 'check'.
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[2]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
 /usr/bin/mkdir -p '/home/user/actest/example-1/hello-1.0/_inst/lib'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libhello.la '/home/user/actest/example-1/hello-1.0/_inst/lib'
libtool: install: /usr/bin/install -c .libs/libhello.so.0.0.0 /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so.0.0.0
libtool: install: (cd /home/user/actest/example-1/hello-1.0/_inst/lib && { ln -s -f libhello.so.0.0.0 libhello.so.0 || { rm -f libhello.so.0 && ln -s libhello.so.0.0.0 libhello.so.0; }; })
libtool: install: (cd /home/user/actest/example-1/hello-1.0/_inst/lib && { ln -s -f libhello.so.0.0.0 libhello.so || { rm -f libhello.so && ln -s libhello.so.0.0.0 libhello.so; }; })
libtool: install: /usr/bin/install -c .libs/libhello.lai /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.la
libtool: install: /usr/bin/install -c .libs/libhello.a /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
libtool: install: chmod 644 /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
libtool: install: ranlib /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
libtool: finish: PATH="/home/hipersayan_x/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/android-ndk:/opt/android-sdk/platform-tools:/opt/android-sdk/tools:/opt/android-sdk/tools/bin:/home/hipersayan_x/.local/share/flatpak/exports/bin:/var/lib/flatpak/exports/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/opt/depot_tools:/sbin" ldconfig -n /home/user/actest/example-1/hello-1.0/_inst/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /home/user/actest/example-1/hello-1.0/_inst/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /usr/bin/mkdir -p '/home/user/actest/example-1/hello-1.0/_inst/bin'
  /bin/sh ./libtool   --mode=install /usr/bin/install -c hello '/home/user/actest/example-1/hello-1.0/_inst/bin'
libtool: install: /usr/bin/install -c .libs/hello /home/user/actest/example-1/hello-1.0/_inst/bin/hello
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Nothing to be done for 'installcheck'.
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
 ( cd '/home/user/actest/example-1/hello-1.0/_inst/bin' && rm -f hello )
 /bin/sh ./libtool   --mode=uninstall rm -f '/home/user/actest/example-1/hello-1.0/_inst/lib/libhello.la'
libtool: uninstall: rm -f /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.la /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so.0.0.0 /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so.0 /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so /home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[2]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
 /usr/bin/mkdir -p '/tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libhello.la '/tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib'
libtool: install: /usr/bin/install -c .libs/libhello.so.0.0.0 /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so.0.0.0
libtool: install: (cd /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib && { ln -s -f libhello.so.0.0.0 libhello.so.0 || { rm -f libhello.so.0 && ln -s libhello.so.0.0.0 libhello.so.0; }; })
libtool: install: (cd /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib && { ln -s -f libhello.so.0.0.0 libhello.so || { rm -f libhello.so && ln -s libhello.so.0.0.0 libhello.so; }; })
libtool: install: /usr/bin/install -c .libs/libhello.lai /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.la
libtool: install: /usr/bin/install -c .libs/libhello.a /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
libtool: install: chmod 644 /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
libtool: install: ranlib /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
libtool: warning: remember to run 'libtool --finish /home/user/actest/example-1/hello-1.0/_inst/lib'
 /usr/bin/mkdir -p '/tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/bin'
  /bin/sh ./libtool   --mode=install /usr/bin/install -c hello '/tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/bin'
libtool: warning: 'libhello.la' has not been installed in '/home/user/actest/example-1/hello-1.0/_inst/lib'
libtool: install: /usr/bin/install -c .libs/hello /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/bin/hello
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
 ( cd '/tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/bin' && rm -f hello )
 /bin/sh ./libtool   --mode=uninstall rm -f '/tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.la'
libtool: uninstall: rm -f /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.la /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so.0.0.0 /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so.0 /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.so /tmp/am-dc-11028//home/user/actest/example-1/hello-1.0/_inst/lib/libhello.a
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make  dist-gzip am__post_remove_distdir='@:'
make[2]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make  distdir-am
make[3]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
if test -d "hello-1.0"; then find "hello-1.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "hello-1.0" || { sleep 5 && rm -rf "hello-1.0"; }; else :; fi
test -d "hello-1.0" || mkdir "hello-1.0"
test -n "" \
|| find "hello-1.0" -type d ! -perm -755 \
        -exec chmod u+rwx,go+rx {} \; -o \
  ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
  ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
  ! -type d ! -perm -444 -exec /bin/sh /home/user/actest/example-1/hello-1.0/build-aux/install-sh -c -m a+r {} {} \; \
|| chmod -R a+r "hello-1.0"
make[3]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
tardir=hello-1.0 && ${TAR-tar} chof - "$tardir" | eval GZIP= gzip --best -c >hello-1.0.tar.gz
make[2]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
if test -d "hello-1.0"; then find "hello-1.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "hello-1.0" || { sleep 5 && rm -rf "hello-1.0"; }; else :; fi
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
make[1]: Entering directory '/home/user/actest/example-1/hello-1.0/_build/sub'
 rm -f hello
test -z "libhello.la" || rm -f libhello.la
rm -f ./so_locations
rm -rf .libs _libs
rm -f *.o
rm -f *.lo
rm -f *.tab.c
test -z "" || rm -f 
test . = "../.." || test -z "" || rm -f 
rm -f libtool config.lt
rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags
rm -f cscope.out cscope.in.out cscope.po.out cscope.files
rm -f config.status config.cache config.log configure.lineno config.status.lineno
rm -f ./.deps/hello.Plo
rm -f ./.deps/main.Po
rm -f Makefile
make[1]: Leaving directory '/home/user/actest/example-1/hello-1.0/_build/sub'
if test -d "hello-1.0"; then find "hello-1.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "hello-1.0" || { sleep 5 && rm -rf "hello-1.0"; }; else :; fi
===========================================
hello-1.0 archives ready for distribution: 
hello-1.0.tar.gz
===========================================
ggouaillardet commented 8 months ago

Tarballs from GitHub are automatically generated and should not be used.

rhc54 commented 8 months ago

In particular, the problem with the GitHub tarballs is that they are missing the submodules used by OMPI. This always leads to the macro errors you report. I've had identical reports at times on my projects, and invariably it turns out that they used the GitHub tarball instead of the official one.

You really need to avoid the GitHub tarballs.

jsquyres commented 8 months ago

I think @rhc54 hit the nail on the head:

You really need to avoid the GitHub tarballs.

I forgot about the issue of those tarballs not including the Git submodules. That will definitely be a problem. When I download https://github.com/open-mpi/ompi/archive/refs/tags/v5.0.2.tar.gz and run autogen.pl on it, I get:

...snipped...

autoreconf: running: /usr/local/Cellar/autoconf/2.71/bin/autoconf --include=config --include=config/oac --force
configure.ac:87: error: possibly undefined macro: AC_MSG_ERROR
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:98: error: possibly undefined macro: AC_MSG_WARN
configure.ac:123: error: possibly undefined macro: AC_MSG_CHECKING
configure.ac:124: error: possibly undefined macro: AC_MSG_RESULT
configure.ac:388: error: possibly undefined macro: AC_COMPILE_IFELSE
configure.ac:389: error: possibly undefined macro: AC_LANG_SOURCE
configure.ac:533: error: possibly undefined macro: AC_LANG_PUSH
configure.ac:534: error: possibly undefined macro: AC_LANG_PROGRAM
configure.ac:901: error: possibly undefined macro: AC_LINK_IFELSE
configure:19150: error: possibly undefined macro: m4_if
configure:19155: error: possibly undefined macro: AS_VAR_SET
configure:19166: error: possibly undefined macro: AC_LANG_POP
configure:55159: error: possibly undefined macro: AS_VAR_COPY
autoreconf: error: /usr/local/Cellar/autoconf/2.71/bin/autoconf failed with exit status: 1
Command failed: autoreconf -ivf --warnings=all,no-obsolete,no-override -I config -I config/oac

It's because the config/oac git submodule is not included in the Github-generated tarball.

Use a properly-bootstrapped, official release tarball, and it should work: https://download.open-mpi.org/release/open-mpi/v5.0/openmpi-5.0.2.tar.bz2

jsquyres commented 8 months ago

I just filed #12402 to update the docs to explicitly mention that GitHub.com-generated tarballs are not official releases and will not build properly.

hipersayanX commented 8 months ago

Well after trying the cloning and initializing the submodules, I have found that config/oac was missing, and autogen.pl successfully passed :smile:

hipersayanX commented 8 months ago

@jsquyres

I think @rhc54 hit the nail on the head:

You really need to avoid the GitHub tarballs.

Yeah I know, but sometimes the official mirror can be blocked (talking in general), and you need to download from another one.

I just filed https://github.com/open-mpi/ompi/pull/12402 to update the docs to explicitly mention that GitHub.com-generated tarballs are not official releases and will not build properly.

And how about providing the full source package through github releases? many projects do that.

rhc54 commented 8 months ago

Yeah I know, but sometimes the official mirror can be blocked (talking in general), and you need to download from another one.

Your better bet in those circumstances is to use a Git clone and just initialize the submodules prior to building. Using GitHub tarballs is almost always a recipe for trouble.

And how about providing the full source package through github releases? many projects do that.

I do that on my projects, but the negative is that it can cause confusion when people see both mine and the GitHub tarballs both posted there. Sadly, GitHub does not provide a way to disable theirs.

rhc54 commented 8 months ago

@jsquyres Perhaps another solution to help identify the problem is to have configure.ac check for the submodules to be populated? We do it in autogen.pl and generate a nice error message if not found - but maybe we need it in configure as well to deal with the GitHub tarball problem?

jsquyres commented 8 months ago

+1 on what @rhc54 said about Github releases. The lengthier explanation is that we were releasing on www.open-mpi.org long before we hosted Open MPI on github.com, and have just kinda kept on doing that. We did also use Github releases for a while, but had both problems that Ralph cited: people would download the Github tarball (and ignore the tarball that we attached to the release), and/or would ask about the difference between the www.open-mpi.org release and the github release. Soooo... we just kinda stopped doing the github releases and kept putting them on www.open-mpi.org

The www.open-mpi.org releases actually go through AWS's CDN (via download.open-mpi.org) -- have you ever had a problem with it being blocked for you?

@rhc54 I guess we could check in configure, but if you download a tarball from github.com, you still have to run autogen.pl, and we should catch it there. Is there a problem case where you won't have had to run autogen.pl?

rhc54 commented 8 months ago

I guess we could check in configure, but if you download a tarball from github.com, you still have to run autogen.pl, and we should catch it there. Is there a problem case where you won't have had to run autogen.pl?

Yeah, oddly enough. If I simply download a GitHub tarball and run ./autogen.pl - not doing anything else in advance - here is what I get:

<snip>

2. Checking for git submodules

fatal: not a git repository (or any of the parent directories): .git

3. Searching for MCA frameworks and components

<snip>
autoreconf: running: /opt/local/bin/automake --add-missing --copy --force-missing
configure.ac:211: installing './config/ar-lib'
configure.ac:113: installing './config/compile'
configure.ac:98: installing './config/config.guess'
configure.ac:98: installing './config/config.sub'
configure.ac:101: installing './config/install-sh'
configure.ac:101: installing './config/missing'
docs/Makefile.am:97: error: PMIX_BUILD_DOCS does not appear in AM_CONDITIONAL
docs/Makefile.am:148: error: PMIX_INSTALL_DOCS does not appear in AM_CONDITIONAL
examples/Makefile.am: installing './config/depcomp'
configure.ac: installing './config/ylwrap'
parallel-tests: installing './config/test-driver'
autoreconf: error: /opt/local/bin/automake failed with exit status: 1
Command failed: autoreconf -ivf --warnings=all,no-obsolete,no-override -I config -I config/oac

So the check for submodule doesn't cause us to exit, even though it reports the error. Not sure how/why that happens, but it clearly happened to this user too.

jsquyres commented 8 months ago

I've updated the docs, and made the Git submodule checks stronger in autogen.pl (such that it will fail if you run autogen.pl in a Github-generated tarball). I think that's all we can do.

Closing this issue.