search

microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
23.27k stars 6.42k forks source link

[openssl] Build error on wasm32-emscripten (Host: x64-windows) #37069

Closed Dykson closed 8 months ago

Dykson commented 8 months ago

Package: openssl:wasm32-emscripten@3.2.1

Host Environment

To Reproduce

vcpkg install Failure logs

-- Using cached openssl-openssl-openssl-3.2.1.tar.gz.
-- Cleaning sources at C:/vcpkg/buildtrees/openssl/src/nssl-3.2.1-8707343470.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source C:/vcpkg/downloads/openssl-openssl-openssl-3.2.1.tar.gz
-- Applying patch script-prefix.patch
-- Applying patch windows/install-layout.patch
-- Applying patch windows/install-pdbs.patch
-- Applying patch unix/android-cc.patch
-- Applying patch unix/move-openssldir.patch
-- Applying patch unix/no-empty-dirs.patch
-- Applying patch unix/no-static-libs-for-shared.patch
-- Using source at C:/vcpkg/buildtrees/openssl/src/nssl-3.2.1-8707343470.clean
-- Using cached msys2-make-4.4.1-1-x86_64.pkg.tar.zst.
-- Using cached msys2-perl-5.38.2-1-x86_64.pkg.tar.zst.
-- Using cached msys2-bash-5.2.026-1-x86_64.pkg.tar.zst.
-- Using cached msys2-coreutils-8.32-5-x86_64.pkg.tar.zst.
-- Using cached msys2-file-5.45-1-x86_64.pkg.tar.zst.
-- Using cached msys2-gawk-5.3.0-1-x86_64.pkg.tar.zst.
-- Using cached msys2-grep-1~3.0-6-x86_64.pkg.tar.zst.
-- Using cached msys2-gzip-1.13-1-x86_64.pkg.tar.zst.
-- Using cached msys2-diffutils-3.10-1-x86_64.pkg.tar.zst.
-- Using cached msys2-pkgconf-2.1.0-1-x86_64.pkg.tar.zst.
-- Using cached msys2-sed-4.9-1-x86_64.pkg.tar.zst.
-- Using cached msys2-msys2-runtime-3.4.10-4-x86_64.pkg.tar.zst.
-- Using cached msys2-libintl-0.22.4-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libxcrypt-4.4.36-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libiconv-1.17-1-x86_64.pkg.tar.zst.
-- Using cached msys2-gmp-6.3.0-1-x86_64.pkg.tar.zst.
-- Using cached msys2-gcc-libs-13.2.0-2-x86_64.pkg.tar.zst.
-- Using cached msys2-libbz2-1.0.8-4-x86_64.pkg.tar.zst.
-- Using cached msys2-liblzma-5.4.6-2-x86_64.pkg.tar.zst.
-- Using cached msys2-libzstd-1.5.5-1-x86_64.pkg.tar.zst.
-- Using cached msys2-zlib-1.3.1-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libreadline-8.2.010-1-x86_64.pkg.tar.zst.
-- Using cached msys2-mpfr-4.2.1-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libpcre-8.45-4-x86_64.pkg.tar.zst.
-- Using cached msys2-ncurses-6.4-2-x86_64.pkg.tar.zst.
-- Using msys root at C:/vcpkg/downloads/tools/msys2/9496971bed5faa54
-- Found external ninja('1.11.1').
-- Getting CMake variables for wasm32-emscripten
-- Getting CMake variables for wasm32-emscripten
-- Using cached msys2-autoconf-wrapper-20221207-1-any.pkg.tar.zst.
-- Using cached msys2-automake-wrapper-20221207-2-any.pkg.tar.zst.
-- Using cached msys2-binutils-2.42-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libtool-2.4.7-3-x86_64.pkg.tar.zst.
-- Using cached msys2-make-4.4.1-1-x86_64.pkg.tar.zst.
-- Using cached msys2-pkgconf-2.1.0-1-x86_64.pkg.tar.zst.
-- Using cached msys2-which-2.21-4-x86_64.pkg.tar.zst.
-- Using cached msys2-bash-5.2.026-1-x86_64.pkg.tar.zst.
-- Using cached msys2-coreutils-8.32-5-x86_64.pkg.tar.zst.
-- Using cached msys2-file-5.45-1-x86_64.pkg.tar.zst.
-- Using cached msys2-gawk-5.3.0-1-x86_64.pkg.tar.zst.
-- Using cached msys2-grep-1~3.0-6-x86_64.pkg.tar.zst.
-- Using cached msys2-gzip-1.13-1-x86_64.pkg.tar.zst.
-- Using cached msys2-diffutils-3.10-1-x86_64.pkg.tar.zst.
-- Using cached msys2-sed-4.9-1-x86_64.pkg.tar.zst.
-- Using cached msys2-msys2-runtime-3.4.10-4-x86_64.pkg.tar.zst.
-- Using cached msys2-autoconf2.71-2.71-3-any.pkg.tar.zst.
-- Using cached msys2-automake1.16-1.16.5-1-any.pkg.tar.zst.
-- Using cached msys2-libiconv-1.17-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libintl-0.22.4-1-x86_64.pkg.tar.zst.
-- Using cached msys2-zlib-1.3.1-1-x86_64.pkg.tar.zst.
-- Using cached msys2-tar-1.35-2-x86_64.pkg.tar.zst.
-- Using cached msys2-findutils-4.9.0-3-x86_64.pkg.tar.zst.
-- Using cached msys2-gmp-6.3.0-1-x86_64.pkg.tar.zst.
-- Using cached msys2-gcc-libs-13.2.0-2-x86_64.pkg.tar.zst.
-- Using cached msys2-libbz2-1.0.8-4-x86_64.pkg.tar.zst.
-- Using cached msys2-liblzma-5.4.6-2-x86_64.pkg.tar.zst.
-- Using cached msys2-libzstd-1.5.5-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libreadline-8.2.010-1-x86_64.pkg.tar.zst.
-- Using cached msys2-mpfr-4.2.1-1-x86_64.pkg.tar.zst.
-- Using cached msys2-libpcre-8.45-4-x86_64.pkg.tar.zst.
-- Using cached msys2-m4-1.4.19-2-x86_64.pkg.tar.zst.
-- Using cached msys2-perl-5.38.2-1-x86_64.pkg.tar.zst.
-- Using cached msys2-ncurses-6.4-2-x86_64.pkg.tar.zst.
-- Using cached msys2-libxcrypt-4.4.36-1-x86_64.pkg.tar.zst.
-- Using msys root at C:/vcpkg/downloads/tools/msys2/09133458fed80fcb
-- Using cached msys2-mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst.
-- Using cached msys2-msys2-runtime-3.4.10-4-x86_64.pkg.tar.zst.
-- Using msys root at C:/vcpkg/downloads/tools/msys2/fdbea3694fb5c0d4
-- Configuring wasm32-emscripten-dbg
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:112 (message):
    Command failed: C:/vcpkg/downloads/tools/msys2/09133458fed80fcb/usr/bin/bash.exe --noprofile --norc --debug -c "V=1 ./../src/nssl-3.2.1-8707343470.clean/vcpkg/configure  \"C:/vcpkg/downloads/tools/msys2/9496971bed5faa54/usr/bin/perl.exe\" \"C:/vcpkg/buildtrees/openssl/src/nssl-3.2.1-8707343470.clean/Configure\" \"enable-static-engine\" \"enable-capieng\" \"no-tests\" \"no-docs\" \"no-shared\" \"no-module\" \"no-apps\" \"threads\" \"no-engine\" \"no-asm\" \"no-sse2\" \"no-srtp\" \"--cross-compile-prefix=\" \"--openssldir=/etc/ssl\" \"--libdir=lib\" \"--disable-silent-rules\" \"--verbose\" \"--disable-shared\" \"--enable-static\" \"--debug\" \"--prefix=/D/Repository/sklwatermark/build-watermark-WebAssembly_Qt_6_7_0_multi_threaded-Debug/vcpkg_installed/wasm32-emscripten/debug\""
    Working Directory: C:/vcpkg/buildtrees/openssl/wasm32-emscripten-dbg
    Error code: 1
    See logs for more information:
      C:\vcpkg\buildtrees\openssl\config-wasm32-emscripten-dbg-out.log
      C:\vcpkg\buildtrees\openssl\config-wasm32-emscripten-dbg-err.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_configure_make.cmake:862 (vcpkg_execute_required_process)
  ports/openssl/unix/portfile.cmake:105 (vcpkg_configure_make)
  ports/openssl/portfile.cmake:77 (include)
  scripts/ports.cmake:172 (include)
C:\vcpkg\buildtrees\openssl\config-wasm32-emscripten-dbg-out.log ``` Configuring OpenSSL version 3.2.1 for target MSYS_NT Using os-specific seed configuration ```
C:\vcpkg\buildtrees\openssl\config-wasm32-emscripten-dbg-err.log ``` + C:/vcpkg/downloads/tools/msys2/9496971bed5faa54/usr/bin/perl.exe C:/vcpkg/buildtrees/openssl/src/nssl-3.2.1-8707343470.clean/Configure enable-static-engine enable-capieng no-tests no-docs no-shared no-module no-apps threads no-engine no-asm no-sse2 no-srtp --cross-compile-prefix= --openssldir=/etc/ssl --libdir=lib --debug --prefix=/D/Repository/sklwatermark/build-watermark-WebAssembly_Qt_6_7_0_multi_threaded-Debug/vcpkg_installed/wasm32-emscripten/debug Usage: Configure [no- ...] [enable- ...] [-Dxxx] [-lxxx] [-Lxxx] [-fxxx] [-Kxxx] [no-hw-xxx|no-hw] [[no-]threads] [[no-]thread-pool] [[no-]default-thread-pool] [[no-]shared] [[no-]zlib|zlib-dynamic] [no-asm] [no-egd] [sctp] [386] [--prefix=DIR] [--openssldir=OPENSSLDIR] [--with-xxx[=vvv]] [--config=FILE] os/compiler[:flags] pick os/compiler from: BC-32 BS2000-OSD BSD-aarch64 BSD-armv4 BSD-generic32 BSD-generic64 BSD-ia64 BSD-nodef-generic32 BSD-nodef-generic64 BSD-nodef-ia64 BSD-nodef-sparc64 BSD-nodef-sparcv8 BSD-nodef-x86 BSD-nodef-x86-elf BSD-nodef-x86_64 BSD-ppc BSD-ppc64 BSD-ppc64le BSD-riscv32 BSD-riscv64 BSD-sparc64 BSD-sparcv8 BSD-x86 BSD-x86-elf BSD-x86_64 Cygwin Cygwin-i386 Cygwin-i486 Cygwin-i586 Cygwin-i686 Cygwin-x86 Cygwin-x86_64 DJGPP MPE/iX-gcc OS390-Unix UEFI UEFI-x86 UEFI-x86_64 UWIN VC-CE VC-CLANG-WIN64-CLANGASM-ARM VC-WIN32 VC-WIN32-ARM VC-WIN32-ARM-UWP VC-WIN32-HYBRIDCRT VC-WIN32-ONECORE VC-WIN32-UWP VC-WIN64-ARM VC-WIN64-ARM-UWP VC-WIN64-CLANGASM-ARM VC-WIN64A VC-WIN64A-HYBRIDCRT VC-WIN64A-ONECORE VC-WIN64A-UWP VC-WIN64A-masm VC-WIN64I aix-cc aix-gcc aix64-cc aix64-gcc aix64-gcc-as android-arm android-arm64 android-armeabi android-mips android-mips64 android-x86 android-x86_64 android64 android64-aarch64 android64-mips64 android64-x86_64 bsdi-elf-gcc cc darwin-i386 darwin-i386-cc darwin-ppc darwin-ppc-cc darwin64-arm64 darwin64-arm64-cc darwin64-ppc darwin64-ppc-cc darwin64-x86_64 darwin64-x86_64-cc gcc haiku-x86 haiku-x86_64 hpux-ia64-cc hpux-ia64-gcc hpux-parisc-cc hpux-parisc-gcc hpux-parisc1_1-cc hpux-parisc1_1-gcc hpux64-ia64-cc hpux64-ia64-gcc hpux64-parisc2-cc hpux64-parisc2-gcc hurd-generic32 hurd-generic64 hurd-x86 hurd-x86_64 ios-cross ios-xcrun ios64-cross ios64-xcrun iossimulator-arm64-xcrun iossimulator-i386-xcrun iossimulator-x86_64-xcrun iossimulator-xcrun iphoneos-cross irix-mips3-cc irix-mips3-gcc irix64-mips4-cc irix64-mips4-gcc linux-aarch64 linux-alpha-gcc linux-aout linux-arm64ilp32 linux-armv4 linux-c64xplus linux-elf linux-generic32 linux-generic64 linux-ia64 linux-latomic linux-mips32 linux-mips64 linux-ppc linux-ppc64 linux-ppc64le linux-sparcv8 linux-sparcv9 linux-x32 linux-x86 linux-x86-clang linux-x86-latomic linux-x86_64 linux-x86_64-clang linux32-riscv32 linux32-s390x linux64-loongarch64 linux64-mips64 linux64-riscv64 linux64-s390x linux64-sparcv9 mingw mingw64 nonstop-nse nonstop-nse_64 nonstop-nse_64_put nonstop-nse_g nonstop-nse_g_tandem nonstop-nse_put nonstop-nsv nonstop-nsx nonstop-nsx_64 nonstop-nsx_64_put nonstop-nsx_g nonstop-nsx_g_tandem nonstop-nsx_put sco5-cc sco5-gcc solaris-sparcv7-cc solaris-sparcv7-gcc solaris-sparcv8-cc solaris-sparcv8-gcc solaris-sparcv9-cc solaris-sparcv9-gcc solaris-x86-gcc solaris64-sparcv9-cc solaris64-sparcv9-gcc solaris64-x86_64-cc solaris64-x86_64-gcc tru64-alpha-cc tru64-alpha-gcc uClinux-dist uClinux-dist64 unixware-2.0 unixware-2.1 unixware-7 unixware-7-gcc vms-alpha vms-alpha-p32 vms-alpha-p64 vms-ia64 vms-ia64-p32 vms-ia64-p64 vms-x86_64 vms-x86_64-cross-ia64 vms-x86_64-p32 vms-x86_64-p64 vos-gcc vxworks-mips vxworks-ppc405 vxworks-ppc60x vxworks-ppc750 vxworks-ppc750-debug vxworks-ppc860 vxworks-ppcgen vxworks-simlinux ```

Additional context

vcpkg.json ``` { "dependencies": [ "openssl", "libxml2", "tiff" ] } ```
Dykson commented 8 months ago

there was no such error on the linux host

dg0yt commented 8 months ago

On the linux host, Configure is smart enough to choose target linux-x32. For the windows host, we probably must choose this explicitly. Please test https://github.com/microsoft/vcpkg/pull/37076 on Windows.

Dykson commented 8 months ago

On the linux host, Configure is smart enough to choose target linux-x32. For the windows host, we probably must choose this explicitly. Please test #37076 on Windows.

The configuration is successful, but now there is an error at the build stage

C:\vcpkg\buildtrees\openssl\build-wasm32-emscripten-dbg-err.log ``` The command line is too long. make[1]: *** [Makefile:12323: providers/libdefault.a] Error 1 make[1]: *** Waiting for unfinished jobs.... The command line is too long. make[1]: *** [Makefile:5302: libcrypto.a] Error 1 make: *** [Makefile:2262: build_sw] Error 2 ```
fwcd commented 8 months ago

I've had issues building OpenSSL for wasm32-emscripten on macOS too. The configure script defaults to darwin-arm64, adding -arch arm64 to the cflags, which emcc doesn't understand. Manually patching the generated makefile (as per https://github.com/openssl/openssl/issues/5443#issuecomment-458029264) fixes the build, but obviously this isn't optimal. Will have to test whether #37076 fixes this.

dg0yt commented 8 months ago

I don't think we can easily fix this windows (host) problem. OpenSSL uses its own build system. If you are willing to share the command line, we might look at mitigations, such as using shorter path for some directories,

dg0yt commented 8 months ago

The command line is too long. make[1]: *** [Makefile:12323: providers/libdefault.a] Error 1

In my mingw builds on windows, some libdefault.a archiver command lines are longer than 8192 characters ... but the build doesn't fail. The Makefile uses msys perl and make as provided by vcpkg, and this perl reports the OS in $^O as msys.

I pushed a patch to limit command line length on msys, but I cannot test it.

dg0yt commented 8 months ago

While this is flagged "community triplet", it might also affect the official android triplets if built on windows host. But I can only test with mingw host triplets, and this environment doesn't show this problem. Related: https://github.com/openssl/openssl/issues/19857

dg0yt commented 8 months ago

Reminder: The tentative command line length fix for Windows needs a tester.