config.sub and config.guess are frequently out of date for Autotools packages

[ocaisa]

For many software packages that use Autotools (particularly in the X11 bundle) the files config.sub and config.guess are out of date in the sources and don't support newer architectures such as Power8.

Updating this as a patch is tedious and is not the intended behaviour anyway, it is expected that the user themselves update these files. As a general rule I think we should be automatically updating these files (if they exist) for packages that use ConfigureMake and have Autotools as a dependency.

[ocaisa]

@geimer I couldn't find documentation on an environment variable setting that might solve this, do you know if such a variable exists?

[geimer]

@ocaisa I don't know of any automatic way to update config.sub and config.guess. As far as I know, package maintainers are supposed to ship "recent enough" versions with their tarballs, and if all fails the message tells the user where to find the latest versions and give it a try. One should also not forget that the package is very likely untested on an architecture the configure script doesn't recognize, i.e., you're in unknown territory.

BTW: When Autotools is a dependency (and in most cases it should really be a builddependency), one needs to update the scripts shipped with Autotools, as the autogen.sh or bootstrap script will copy them from the Autotools installation into the package source tree.

[ocaisa]

Oh, thats good to know, so basically if I update the versions found in Automake, I can get a lot of the way there. I'll give it a shot.

[boegel]

@ocaisa Rather than always regenerating config.sub & config.guess, we can add support for it in ConfigureMake so you can do it easily?

If the configuration works fine without regenerating them, there's no reason to? And like @geimer says, it's possible you'll still run into problems regardless when trying to build stuff on uncommon arichitectures...

[geimer]

@boegel These two files are not generated. They are just copied into the source tree and called from the generated configure. The "problem" is that the install location isn't fixed. So one has to find all instances in the source tree and replace them. I believe this should work...

[omula]

I have to install several software on a ppc64le (Power9) and I have already had to replace many config.guess files.

I still have many software pending to install so I'd prefer to work on a solution instead of manually creating patches for each of the software.

@ocaisa do you have any proposal in mind?

[ocaisa]

I really wish there was some envvars we could set to point to config.guess but I couldn't find anything. The only thing I could think of was modifying the easyblock to automatically do a search/replace on the necessary files in the unpacked sources.

I wrote a script that does that, but it is a poor hack.

[omula]

Following the suggestion in https://lists.gnu.org/archive/html/automake/2004-08/msg00032.html running automake --add-missing --copy --force-missing seems to work for replacing the files. However, then other issues appear like:

configure.ac:29: error: version mismatch. This is Automake 1.13.4, configure.ac:29: but the definition used by this AM_INIT_AUTOMAKE configure.ac:29: comes from Automake 1.11.1. You should recreate configure.ac:29: aclocal.m4 with aclocal and run automake again. configure.ac:29: warning: The 'AM_PROG_MKDIR_P' macro is deprecated, and its use is discouraged. configure.ac:29: You should use the Autoconf-provided 'AC_PROG_MKDIR_P' macro instead, configure.ac:29: and use '$(MKDIR_P)' instead of '$(mkdir_p)'in your Makefile.am files.

After some trial and error it worked running before autoreconf so adding a preconfigopts would work. preconfigopts = 'autoreconf && automake --add-missing --copy --force-missing && '

I have made the test for Autoconf-2.69 and as an extra I had to add help2man in buildependencies and a patch https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=711297.

[omula]

I have just tried with LibTIFF which had the same issue and adding the preconfigopts worked. preconfigopts = 'autoreconf && automake --add-missing --copy --force-missing && '

[ocaisa]

That might be the right way to go but we would need that to be set in the configuremake easyblock and trigger it somehow when the patched Autoconf is used.

@boegel Thoughts?

[ocaisa]

This is how Spack handles it: https://github.com/spack/spack/blob/develop/lib/spack/spack/build_systems/autotools.py#L103

[boegel]

Regenerating via autoreconf & automake definitely feels cleaner than hard overwriting config.guess, but I'm not sure what the better option is.

@geimer: Any idea which approach should be preferred?

[omula]

Just to make you aware that the solution that I used with LibTIFF does not always work. So, maybe spacks approach might be a good solution.

numactl-2.0.11-GCCcore-7.3.0.eb:preconfigopts = "./autogen.sh && "
Opt++-2.4-foss-2018b.eb:preconfigopts = 'autoreconf -fmi && '
CppUnit-1.13.2-GCCcore-7.3.0.eb:preconfigopts = 'autoreconf --install && automake --add-missing --copy --force-missing && '

[geimer]

@boegel Running autoreconf can have arbitrary side effects. It is kind of a make for Autotools build systems, i.e., it will regenerate configure scripts, Makefile.in files, etc. if they are out of date. But the project's input files may depend on specific versions of Autoconf, Automake, or libtool, that is, autoreconf-ing with a newer Autotools versions to get updated config.sub and config.guess files may not work.

Now, while such files distributed as part of release tarballs normally shouldn't be out of date, we have seen strange cases where they got similar enough timestamps during tarball extraction which triggered the regeneration of files. We couldn't figure out the exact reason (potentially it's file system specific?); re-extracting the tarball often fixed the issue.

As far as I know, config.sub and config.guess are shell scripts that are just called by configure and the "interface" hasn't changed for a long time. Thus it is probably safe to simply overwrite them with newer versions and keep all the other files alone.

[micaeljtoliveira]

I've recently stumbled on a few out-of-date config.guess scripts when building on a powerpc, so I was wondering if you had reached a consensus about the best way to solve this?

As a quick fix I actually used a different solution from the ones mentioned here. As it happens, one can explicitly set the system name when invoking the configure script by using the --build flag. This way the config.guess script is not needed anymore. So this got me thinking that maybe this could be used by the ConfigureMake easyblock: have the easyblock call an up-to-date version of config.guess then pass the resulting name to the configure script. This way out-dated config.guess scripts are never invoked and there is no need to patch or change in any way the package being built. Would this make sense?

[ocaisa]

I like that solution, we could add a recent config.guess to the framework scripts directory and call out to that and update as necessary

[ocaisa]

Working on a solution in https://github.com/easybuilders/easybuild-easyblocks/pull/1506 , comments welcome!

[ocaisa]

Fixed in #1506