skeeto / w64devkit

Portable C and C++ Development Kit for x64 (and x86) Windows
The Unlicense
2.66k stars 185 forks source link

error: no acceptable grep could be found in #121

Open xubuntu4iran opened 3 months ago

xubuntu4iran commented 3 months ago
~/Downloads/gmp-6.3.0 $ ./configure
checking build system type... configfsf.guess: cannot create a temporary directory in /tmp
configure: error: cannot guess build type; you must specify one
~/Downloads/gmp-6.3.0 $ ./configure --build=x86_64
checking build system type... x86_64-pc-none
checking host system type... x86_64-pc-none
checking for a BSD-compatible install... ./install-sh -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... no
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking ABI=64
checking compiler gcc -O2 -pedantic -fomit-frame-pointer -m64 ... yes
checking compiler gcc -O2 -pedantic -fomit-frame-pointer -m64  -mtune=k8... yes
checking compiler gcc -O2 -pedantic -fomit-frame-pointer -m64 -mtune=k8  -march=k8... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking for gcc option to accept ISO C99... none needed
checking how to run the C preprocessor... gcc -E
checking build system compiler gcc... yes
checking for build system preprocessor... gcc -E
checking for build system executable suffix... .exe
checking whether build system compiler is ANSI... yes
checking for build system compiler math library... -lm
checking for grep that handles long lines and -e... configure: error: no acceptable grep could be found in D:/w64devkit/bin;C:/Windows/system32;C:/Windows;C:/Windows/System32/Wbem;C:/Windows/System32/WindowsPowerShell/v1.0/;C:/Windows/System32/OpenSSH/;C:/Users/m.alipour/AppData/Local/Microsoft/WindowsApps;:/usr/xpg4/bin
skeeto commented 3 months ago

See the discussion on #50. In particular you need PATH_SEPARATOR=";" and ac_executable_extensions=".exe" to mitigate issues in configure. Then to mitigate a libtool bug you'll need one of the "cmd //c" workarounds. After that it successfully builds.

MagicalDrizzle commented 2 months ago

I actually couldn't get configure to work even with all the workarounds...sourcing scripts, fake cmd.exe, "bbconf" without the workarounds it chokes at gcc, with them it chokes at grep -E the configure script is from clustalo 1.2.4 which builds fine on msys2, I'm curious if I'm doing something wrong or if this script really has some other quirks to it configure.txt

skeeto commented 2 months ago

I retried on a fresh install to see if something in my normal environment affected the results and it did: GMP normally requires m4 (!) during the build. I had built and installed GNU m4 awhile back, and GMP had found and used it. So one option is to do the same, which of course requires similar configure script contortions. Alternatively you can disable assembly so that m4 is no longer required. I'll cover that option here.

So first, set the environment variables:

$ export PATH_SEPARATOR=";" $ export ac_executable_extensions=.exe

Next, build and install my fake cmd.exe in w64devkit:

$ cmd -nostartfiles -o "$W64DEVKIT_HOME"/bin/cmd.exe cmd.c

Then for GMP I came up with this:

./configure --build=$(gcc -dumpmachine) --prefix="$W64DEVKIT_HOME"/$(gcc -dumpmachine) --disable-assembly M4=m4-not-needed

A build of "x86_64" is insufficient because GMP assumes LP64, which is incorrect for x64. The actual string will be "x86_64-w64-mingw32", which will not configure as LP64.

For the prefix I installed it inside w64devkit. That makes , -lgmp, and even "pkg-config gmp" work out of the box, just as though it came with w64devkit. However, you can put this wherever you prefer. If you place its "pkgconfig" directory in $PKG_CONFIG_PATH it will work with pkg-config no matter where you put it.

I disabled assembly so that m4 isn't needed. However, there's a GMP bug dating back to at least 2010 such that this does not actually disable the m4 test. So the last part, m4-not-needed, is the conventional workaround. If you have m4 on you path, you can skip these last two parts.

With that I could then "make -j$(nproc)" then "make install" and I had a working GMP.

Though now I've just realized you're asking about an entirely different configure script. That will require similar workarounds, but maybe this could serve as a guide if you're really interested in making it work, though it's probably not worth the trouble. It works on msys2 because, as discussed on the other thread, Autotools hard codes hacks into configure scripts to mitigate msys2 problems (e.g. its behavior converting "cmd /c" into "cmd C:/"), and those hacks break the script for systems which don't have those problems (native Windows). Despite its facade of adapting to the host, in practice Autotools builds only support a handful of systems that were popular at the time the configure script was generated.

avih commented 2 months ago

Next, build and install my fake cmd.exe in w64devkit: $ cmd -nostartfiles -o "$W64DEVKIT_HOME"/bin/cmd.exe cmd.c

Are you referring to this? https://gist.github.com/skeeto/2e3ae0e8623f5ca8ef4adbae9f05449b

I don't think I see it in this repo or in the distribution.

Would it make sense to include it at the distribution, possisbly even precompiled inside bin/?

Would it make sense to export PATH_SEPARATOR, ac_executable_extensions, and build_alias automatically, like you suggested here? (and add a blurb to the readme for those who don't use the main launcher and instead only add bin to their path)

I did notice two things though, both relate to cmd //c ...:

  1. The cmd.exe hack is not always required, e.g. with (pre-generated) configure script of gnu mtools 4.0.43.
  2. The sed -i 's/func_convert_file_msys_to_w32/func_convert_file_noop/' configure mentioned here does not work (and also not required) with the same mtools configure script, because these functions don't exist there.

But the 3 exports are required with the mtools configure. Though the build_alias export can be replaced with a --build=<same-value-as-build_alias> option for ./configure.

skeeto commented 2 months ago

Yup, it's just sitting in that gist for now, mostly as a proof of concept, and I haven't given it further attention. It must compile to cmd.exe, but I wouldn't put this in $PATH in normal circumstances. This inclines me towards a new command through which to run configure scripts that enables workarounds which don't involve manipulating the script text. I don't have name picked out, but imagine something like:

$ fixconf ./configure --prefix=... --enable-shared ...

I'd compile cmd.c to, say, lib/fixconf/cmd.exe, and fixconf would:

  1. Add $W64DEVKIT_HOME/lib/fixconf to the front of $PATH
  2. Set PATH_SEPARATOR, ac_executable_extensions, and (maybe) build_alias
  3. Invoke sh.exe with the original command string and wait for it

Then using this configure prefix would fix most Autoconf script builds invoked within that environment. That includes builds that don't require the msys2 cmd.exe fix, like mtools, because the fix wouldn't affect it.

avih commented 2 months ago

Add $W64DEVKIT_HOME/lib/fixconf to the front of $PATH

Well, preferably relative to fixconf itself, again, in case we're not running via the main launcher.

Otherwise sounds good.

However, I don't think we have enough stats with configure scripts to know the possible gotchas and workarounds other than the cmd.exe hack which is not always required.

I don't frequently run autotools configure in w64devkit, so I don't have such stats.

Also, I think the main issue with requiring build_alias or --build=... is that uname -s returns Windows_NT where config.guess looks for MINGW<not sure if 32/64/nothing matters>.

But because configure is executed as a (busybox) shell script, it means it can load a uname shell script which runs busybox uname "$@" and then modifies the result according to whatever, and that uname script can reside together with the fake cmd.exe in high PATH priority.

It probably wouldn't be very hard to address various issues, but collecting those issues might not be simple.

avih commented 2 months ago

For future reference, the ugly cmd //c ... in expat comes, as far as I can tell, from libtool:

https://github.com/autotools-mirror/libtool/blob/7b0918310776395e1cbafe6a08b8df8f8a8a3e21/build-aux/ltmain.in#L958-L967

This is not exactly the form which the function appears at the expat configure script, but I think it's its source somehow.

(//C is also used here and here but these are only invoked when the compiler is ICC or MSVC CL, so not in w64devkit - I tested it)

Not 100% clear to me when it does or doesn't get into configure (e.g. yes in expat, no with mtools, maybe always when using libtool?), but it's still in libtool today, and can still be part of newly generated configure scripts.

So we can't shrug it off as "only expat".

As a possible fix, and because it's explicitly targetting msys[2], maybe the upstream solution is to replace all the cmd //C ... instances with MSYS2_ARG_CONV_EXCL=* cmd /C .... This would prevent the unix-path-to-dos-path arg conversion in msys2 so that cmd /c ... works, and it will work as epected also when such auto-conversion doesn't happen (busybox-w32 sh).

Just FYI.

As a side note, the cmd.c hack source seems to check //c (lower case), but as far as I can tell, it's there are also cases which use //C (upper case), though apparently not when building expat in w64dk.