various Makefile.am missing $(EXEEXT) for running programs at compile time

[claudeha]

Context: cross-compiling from Debian Buster Linux x86_64 (with i686 multiarch and Wine) to Microsoft Windows x86_64 and i686 using MINGW W64.

./configure reports checking for suffix of executables... .exe but this information is not used when invoking ./eLut (which is really in ./eLut.exe), thus the command is not found and the build fails.

AFAICT the solution is to put in $(EXEEXT) in all the Makefile.am everywhere after the names of programs that are run at compile time, for example ./eLut$(EXEEXT). My count is 8 occurrences needed in IlmBase/Half/Makefile.am and 6 in OpenEXR/IlmImf/Makefile.am.

$ cd ~/code/github.com/openexr/openexr/IlmBase
$ ./bootstrap 
running aclocal -I m4 ...
running libtoolize --automake --copy ...
running automake --add-missing --copy --foreign ...
configure.ac:33: installing './compile'
configure.ac:16: installing './config.guess'
configure.ac:16: installing './config.sub'
configure.ac:20: installing './install-sh'
configure.ac:20: installing './missing'
Half/Makefile.am: installing './depcomp'
parallel-tests: installing './test-driver'
running autoconf ...

Now type './configure' to configure IlmBase.

$ ./configure --host=x86_64-w64-mingw32
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-w64-mingw32
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for x86_64-w64-mingw32-strip... x86_64-w64-mingw32-strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for x86_64-w64-mingw32-g++... x86_64-w64-mingw32-g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether x86_64-w64-mingw32-g++ accepts -g... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of x86_64-w64-mingw32-g++... gcc3
checking for x86_64-w64-mingw32-gcc... x86_64-w64-mingw32-gcc
checking whether we are using the GNU C compiler... yes
checking whether x86_64-w64-mingw32-gcc accepts -g... yes
checking for x86_64-w64-mingw32-gcc option to accept ISO C89... none needed
checking whether x86_64-w64-mingw32-gcc understands -c and -o together... yes
checking dependency style of x86_64-w64-mingw32-gcc... gcc3
checking whether ln -s works... yes
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by x86_64-w64-mingw32-gcc... /usr/bin/x86_64-w64-mingw32-ld
checking if the linker (/usr/bin/x86_64-w64-mingw32-ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/x86_64-w64-mingw32-nm -B
checking the name lister (/usr/bin/x86_64-w64-mingw32-nm -B) interface... BSD nm
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-w64-mingw32 format... func_convert_file_nix_to_w32
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/x86_64-w64-mingw32-ld option to reload object files... -r
checking for x86_64-w64-mingw32-objdump... x86_64-w64-mingw32-objdump
checking how to recognize dependent libraries... file_magic ^x86 archive import|^x86 DLL
checking for x86_64-w64-mingw32-dlltool... x86_64-w64-mingw32-dlltool
checking how to associate runtime and link libraries... func_cygming_dll_for_implib
checking for x86_64-w64-mingw32-ar... x86_64-w64-mingw32-ar
checking for archiver @FILE support... @
checking for x86_64-w64-mingw32-strip... (cached) x86_64-w64-mingw32-strip
checking for x86_64-w64-mingw32-ranlib... x86_64-w64-mingw32-ranlib
checking command to parse /usr/bin/x86_64-w64-mingw32-nm -B output from x86_64-w64-mingw32-gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for x86_64-w64-mingw32-mt... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... x86_64-w64-mingw32-gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... no
checking for objdir... .libs
checking if x86_64-w64-mingw32-gcc supports -fno-rtti -fno-exceptions... no
checking for x86_64-w64-mingw32-gcc option to produce PIC... -DDLL_EXPORT -DPIC
checking if x86_64-w64-mingw32-gcc PIC flag -DDLL_EXPORT -DPIC works... yes
checking if x86_64-w64-mingw32-gcc static flag -static works... yes
checking if x86_64-w64-mingw32-gcc supports -c -o file.o... yes
checking if x86_64-w64-mingw32-gcc supports -c -o file.o... (cached) yes
checking whether the x86_64-w64-mingw32-gcc linker (/usr/bin/x86_64-w64-mingw32-ld) supports shared libraries... yes
checking whether -lc should be explicitly linked in... yes
checking dynamic linker characteristics... Win32 ld.exe
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... x86_64-w64-mingw32-g++ -E
checking for ld used by x86_64-w64-mingw32-g++... /usr/bin/x86_64-w64-mingw32-ld
checking if the linker (/usr/bin/x86_64-w64-mingw32-ld) is GNU ld... yes
checking whether the x86_64-w64-mingw32-g++ linker (/usr/bin/x86_64-w64-mingw32-ld) supports shared libraries... yes
checking for x86_64-w64-mingw32-g++ option to produce PIC... -DDLL_EXPORT -DPIC
checking if x86_64-w64-mingw32-g++ PIC flag -DDLL_EXPORT -DPIC works... yes
checking if x86_64-w64-mingw32-g++ static flag -static works... yes
checking if x86_64-w64-mingw32-g++ supports -c -o file.o... yes
checking if x86_64-w64-mingw32-g++ supports -c -o file.o... (cached) yes
checking whether the x86_64-w64-mingw32-g++ linker (/usr/bin/x86_64-w64-mingw32-ld) supports shared libraries... yes
checking dynamic linker characteristics... Win32 ld.exe
checking how to hardcode library paths into programs... immediate
checking whether make sets $(MAKE)... (cached) yes
checking whether x86_64-w64-mingw32-g++ supports C++14 features by default... yes
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for cc_r... x86_64-w64-mingw32-gcc
checking semaphore.h usability... yes
checking semaphore.h presence... yes
checking for semaphore.h... yes
checking for library containing sem_init... none required
checking whether to use POSIX unnamed semaphores... no (pshared not usable)
configure: multithread true, LIBS = -pthread , CC = x86_64-w64-mingw32-gcc, CXXFLAGS = -g -O2 -std=c++14 -pthread
checking for ANSI C header files... (cached) yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking for unistd.h... (cached) yes
checking ucontext.h usability... no
checking ucontext.h presence... no
checking for ucontext.h... no
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking for size_t... yes
checking for strerror... yes
checking for complete iomanip support in C++ standard library... yes
checking for fpe support handling... no
checking for gcc optimization flags... -pipe
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating IlmBase.pc
config.status: creating config/Makefile
config.status: creating Half/Makefile
config.status: creating HalfTest/Makefile
config.status: creating Iex/Makefile
config.status: creating IexMath/Makefile
config.status: creating IexTest/Makefile
config.status: creating IlmThread/Makefile
config.status: creating Imath/Makefile
config.status: creating ImathTest/Makefile
config.status: creating config/IlmBaseConfig.h
config.status: creating config/IlmBaseConfigInternal.h
config.status: executing depfiles commands
config.status: executing libtool commands

---------------------------------------------
Summary for IlmBase features:
enable large stack optimizations             no
ISO C++ Standard                             14

internal library namespaces                  Imath_2_3
                                             Iex_2_3
                                             IlmThread_2_3

public namespaces                            Imath Iex IlmThread

---------------------------------------------

$ make
Making all in Half
make[1]: Entering directory '/home/claude/code/github.com/openexr/openexr/IlmBase/Half'
x86_64-w64-mingw32-g++ -g -O2 -std=c++14 -pthread    eLut.cpp   -o eLut
./eLut > eLut.h
/bin/bash: ./eLut: No such file or directory
make[1]: *** [Makefile:771: eLut.h] Error 127
make[1]: Leaving directory '/home/claude/code/github.com/openexr/openexr/IlmBase/Half'
make: *** [Makefile:470: all-recursive] Error 1
$ git describe
v2.1.0-463-gcd1b068

[kdt3rd]

We are definitely focusing more effort on making sure the cmake path of compiling works in the current master, and going forward we may deprecate and remove the automake stuff, since the cmake path works to generate builds for more systems. That being said, what you say sounds like the correct change.

But if you could give a go at cross compiling the current master branch using the cmake toolchain mechanism (see the comment at the top of cmake/Toolchain-mingw.cmake), and let me know success / failure, I would appreciate it as I've only tested this under Arch. You may need to edit the file if you have different paths, or want different flags, and that is expected given it's a toolchain file.

One thing the cmake mechanism provides is the CMAKE_CROSSCOMPILING_EMULATOR setting, which can be set to wine, in case PE binaries are not registered as a binary type to launch with wine, or if you need to run an emulator. Do you think that would be needed as well for the automake setup?

[claudeha]

It fails with a separate problem:

[  8%] Building CXX object IlmBase/IlmThread/CMakeFiles/IlmThread.dir/IlmThread.cpp.obj
In file included from /home/claude/tmp/bug544/openexr/IlmBase/IlmThread/IlmThread.cpp:45:
/home/claude/tmp/bug544/openexr/IlmBase/IlmThread/IlmThread.h:143:10: error: ‘thread’ in namespace ‘std’ does not name a type
     std::thread _thread;
          ^~~~~~

I have successfully built it by hacking out the #include <thread> and #include <mutex> and replacing them with references to the header-only library https://github.com/meganz/mingw-std-threads (by editing the source code to the full paths on my system).

[claudeha]

$ x86_64-w64-mingw32-g++ --version x86_64-w64-mingw32-g++ (GCC) 8.3-win32 20190406

Debian Buster.

[cary-ilm]

I'm happy to add the $(EXEEXT) to the Makefile.am's, although you mention 6 and 8 references, I only see 2 in each file - eLut and toFloat in Half and b44ExpLogTable and dwaLookups in IlmImf, what am I missing?

I do notice that the automake-generated Makefiles already append the $(EXEEXT) to the program names, so it should be just the invocation lines in Makefile.am, right?

It's worth noting that our plan is to eliminate this build-time code-generation step from the build process entirely in the next release, because it's so fragile.

[claudeha]

On 05/09/2019 18:08, Cary Phillips wrote:

I'm happy to add the $(EXEEXT) to the Makefile.am's, although you mention 6 and 8 references, I only see 2 in each file - eLut and toFloat in Half and b44ExpLogTable and dwaLookups in IlmImf, what am I missing?

for each program: ./invocation,  CLEANFILES, right hand side of rule dependencies, noinst_PROGRAMS - I added it to all of those, seemed to do no harm, don't know if all are necessary or correct.

I do notice that the automake-generated Makefiles already append the $(EXEEXT) to the program names, so it should be just the invocation lines in Makefile.am, right?

I'm no automake expert, I just hacked this stuff to make it build (with the previous release archives).  And if the autoconf build system is deprecated in favour of cmake I don't know whether it's worth to fix it before it goes away.

It's worth noting that our plan is to eliminate this build-time code-generation step from the build process entirely in the next release, because it's so fragile.

Great.

[kdt3rd]

The <thread> and <mutex> are part of libstdc++ and c++11, which says your gcc / g++ install is not complete for mingw, I have no such issues with those headers under arch.

[cary-ilm]

We've now deprecated the build-time header generation entirely, the headers are now regular source files, to avoid this very kind of issue. Hopefully this resolves the matter.