search

lwthiker / curl-impersonate

curl-impersonate: A special build of curl that can impersonate Chrome & Firefox
MIT License
3.45k stars 229 forks source link

libidn2 error: Library not loaded #182

Closed Explosion-Scratch closed 3 months ago

Explosion-Scratch commented 9 months ago 
dyld[91021]: Library not loaded: /usr/local/opt/libidn2/lib/libidn2.0.dylib
  Referenced from: <822EF38A-1AB2-3BFC-A3FE-92E2005CA7C5> /Users/me/Documents/mydir/curl-impersonate-chrome
  Reason: tried: '/opt/homebrew/Cellar/libidn2/2.3.4_1/lib//libidn2.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/libidn2.0.dylib' (no such file), '/usr/local/opt/libidn2/lib/libidn2.0.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/libidn2/lib/libidn2.0.dylib' (no such file), '/usr/local/opt/libidn2/lib/libidn2.0.dylib' (no such file), '/usr/local/lib/libidn2.0.dylib' (no such file), '/usr/lib/libidn2.0.dylib' (no such file, not in dyld cache)
[1]    91021 abort      ./curl-impersonate-chrome
Explosion-Scratch commented 9 months ago

Building from the source is also failing:

Full log ``` zsh: correct 'gmake' to 'make' [nyae]? n curl -L "https://github.com/google/brotli/archive/refs/tags/v1.0.9.tar.gz" \ -o "brotli-1.0.9.tar.gz" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 475k 0 475k 0 0 682k 0 --:--:-- --:--:-- --:--:-- 682k tar xf brotli-1.0.9.tar.gz cd brotli-1.0.9 mkdir -p out cd out # Convert autoconf style os name to CMake style os name. case darwin23.0.0 in \ linux*) \ system_name=Linux \ ;; \ darwin*) \ system_name=Darwin \ ;; \ *) \ system_name=darwin23.0.0 \ ;; \ esac cmake -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=./installed \ -DCMAKE_INSTALL_LIBDIR=lib \ -DCMAKE_CXX_COMPILER=g++ \ -DCMAKE_C_COMPILER=gcc \ -DCMAKE_SYSTEM_NAME=$system_name \ -DCMAKE_SYSTEM_PROCESSOR=aarch64 \ .. cmake --build . --config Release --target install CMake Deprecation Warning at CMakeLists.txt:5 (cmake_minimum_required): Compatibility with CMake < 3.5 will be removed from a future version of CMake. Update the VERSION argument value or use a ... suffix to tell CMake that the project does not need compatibility with older versions. -- The C compiler identification is AppleClang 15.0.0.15000040 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Build type is 'Release' -- Performing Test BROTLI_EMSCRIPTEN -- Performing Test BROTLI_EMSCRIPTEN - Failed -- Compiler is not EMSCRIPTEN -- Looking for log2 -- Looking for log2 - found CMake Warning at CMakeLists.txt:319 (message): Test file tests/testdata/alice29.txt does not exist. CMake Warning at CMakeLists.txt:319 (message): Test file tests/testdata/asyoulik.txt does not exist. CMake Warning at CMakeLists.txt:319 (message): Test file tests/testdata/lcet10.txt does not exist. CMake Warning at CMakeLists.txt:319 (message): Test file tests/testdata/plrabn12.txt does not exist. -- Configuring done (0.8s) CMake Warning (dev): Policy CMP0042 is not set: MACOSX_RPATH is enabled by default. Run "cmake --help-policy CMP0042" for policy details. Use the cmake_policy command to set the policy and suppress this warning. MACOSX_RPATH is not specified for the following targets: brotlicommon brotlidec brotlienc This warning is for project developers. Use -Wno-dev to suppress it. -- Generating done (0.1s) -- Build files have been written to: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out gmake[1]: Entering directory '/Users/me/Documents/mydir/cli/curl/curl_impersonate_build/curl-impersonate/build/brotli-1.0.9/out' [ 1%] Building C object CMakeFiles/brotlicommon.dir/c/common/constants.c.o [ 3%] Building C object CMakeFiles/brotlicommon.dir/c/common/context.c.o [ 4%] Building C object CMakeFiles/brotlicommon.dir/c/common/dictionary.c.o [ 6%] Building C object CMakeFiles/brotlicommon.dir/c/common/platform.c.o [ 7%] Building C object CMakeFiles/brotlicommon.dir/c/common/transform.c.o [ 9%] Linking C shared library libbrotlicommon.dylib [ 9%] Built target brotlicommon [ 10%] Building C object CMakeFiles/brotlidec.dir/c/dec/bit_reader.c.o [ 12%] Building C object CMakeFiles/brotlidec.dir/c/dec/decode.c.o [ 13%] Building C object CMakeFiles/brotlidec.dir/c/dec/huffman.c.o [ 15%] Building C object CMakeFiles/brotlidec.dir/c/dec/state.c.o [ 16%] Linking C shared library libbrotlidec.dylib [ 16%] Built target brotlidec [ 18%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references.c.o [ 19%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references_hq.c.o [ 21%] Building C object CMakeFiles/brotlienc.dir/c/enc/bit_cost.c.o [ 22%] Building C object CMakeFiles/brotlienc.dir/c/enc/block_splitter.c.o [ 24%] Building C object CMakeFiles/brotlienc.dir/c/enc/brotli_bit_stream.c.o [ 25%] Building C object CMakeFiles/brotlienc.dir/c/enc/cluster.c.o [ 27%] Building C object CMakeFiles/brotlienc.dir/c/enc/command.c.o [ 28%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment.c.o [ 30%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment_two_pass.c.o [ 31%] Building C object CMakeFiles/brotlienc.dir/c/enc/dictionary_hash.c.o [ 33%] Building C object CMakeFiles/brotlienc.dir/c/enc/encode.c.o [ 34%] Building C object CMakeFiles/brotlienc.dir/c/enc/encoder_dict.c.o [ 36%] Building C object CMakeFiles/brotlienc.dir/c/enc/entropy_encode.c.o [ 37%] Building C object CMakeFiles/brotlienc.dir/c/enc/fast_log.c.o [ 39%] Building C object CMakeFiles/brotlienc.dir/c/enc/histogram.c.o [ 40%] Building C object CMakeFiles/brotlienc.dir/c/enc/literal_cost.c.o [ 42%] Building C object CMakeFiles/brotlienc.dir/c/enc/memory.c.o [ 43%] Building C object CMakeFiles/brotlienc.dir/c/enc/metablock.c.o [ 45%] Building C object CMakeFiles/brotlienc.dir/c/enc/static_dict.c.o [ 46%] Building C object CMakeFiles/brotlienc.dir/c/enc/utf8_util.c.o [ 48%] Linking C shared library libbrotlienc.dylib [ 48%] Built target brotlienc [ 50%] Building C object CMakeFiles/brotlicommon-static.dir/c/common/constants.c.o [ 51%] Building C object CMakeFiles/brotlicommon-static.dir/c/common/context.c.o [ 53%] Building C object CMakeFiles/brotlicommon-static.dir/c/common/dictionary.c.o [ 54%] Building C object CMakeFiles/brotlicommon-static.dir/c/common/platform.c.o [ 56%] Building C object CMakeFiles/brotlicommon-static.dir/c/common/transform.c.o [ 57%] Linking C static library libbrotlicommon-static.a [ 57%] Built target brotlicommon-static [ 59%] Building C object CMakeFiles/brotlidec-static.dir/c/dec/bit_reader.c.o [ 60%] Building C object CMakeFiles/brotlidec-static.dir/c/dec/decode.c.o [ 62%] Building C object CMakeFiles/brotlidec-static.dir/c/dec/huffman.c.o [ 63%] Building C object CMakeFiles/brotlidec-static.dir/c/dec/state.c.o [ 65%] Linking C static library libbrotlidec-static.a [ 65%] Built target brotlidec-static [ 66%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/backward_references.c.o [ 68%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/backward_references_hq.c.o [ 69%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/bit_cost.c.o [ 71%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/block_splitter.c.o [ 72%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/brotli_bit_stream.c.o [ 74%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/cluster.c.o [ 75%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/command.c.o [ 77%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/compress_fragment.c.o [ 78%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/compress_fragment_two_pass.c.o [ 80%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/dictionary_hash.c.o [ 81%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/encode.c.o [ 83%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/encoder_dict.c.o [ 84%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/entropy_encode.c.o [ 86%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/fast_log.c.o [ 87%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/histogram.c.o [ 89%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/literal_cost.c.o [ 90%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/memory.c.o [ 92%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/metablock.c.o [ 93%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/static_dict.c.o [ 95%] Building C object CMakeFiles/brotlienc-static.dir/c/enc/utf8_util.c.o [ 96%] Linking C static library libbrotlienc-static.a [ 96%] Built target brotlienc-static [ 98%] Building C object CMakeFiles/brotli.dir/c/tools/brotli.c.o [100%] Linking C executable brotli [100%] Built target brotli Install the project... -- Install configuration: "Release" -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/bin/brotli -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlienc.1.0.9.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlienc.1.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlienc.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlidec.1.0.9.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlidec.1.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlidec.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlicommon.1.0.9.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlicommon.1.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlicommon.dylib -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlienc-static.a -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlidec-static.a -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/libbrotlicommon-static.a -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/include/brotli -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/include/brotli/port.h -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/include/brotli/types.h -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/include/brotli/encode.h -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/include/brotli/decode.h -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/pkgconfig/libbrotlicommon.pc -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/pkgconfig/libbrotlidec.pc -- Installing: /Users/me/Documents/mydir/curl-impersonate/build/brotli-1.0.9/out/installed/lib/pkgconfig/libbrotlienc.pc gmake[1]: Leaving directory '/Users/me/Documents/mydir/cli/curl/curl_impersonate_build/curl-impersonate/build/brotli-1.0.9/out' curl -L https://github.com/google/boringssl/archive/3a667d10e94186fd503966f5638e134fe9fb4080.zip \ -o boringssl.zip % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 32.9M 0 32.9M 0 0 7549k 0 --:--:-- 0:00:04 --:--:-- 9653k unzip -q -o boringssl.zip mv boringssl-3a667d10e94186fd503966f5638e134fe9fb4080 boringssl cd boringssl/ for p in /Users/me/Documents/mydir/curl-impersonate/build/../chrome/patches/boringssl-old-ciphers.patch; do patch -p1 < $p; done touch .patched patching file 'ssl/internal.h' patching file 'ssl/ssl_cipher.cc' mkdir -p /Users/me/Documents/mydir/curl-impersonate/build/boringssl/build cd /Users/me/Documents/mydir/curl-impersonate/build/boringssl/build # Convert autoconf style os name to CMake style os name. case darwin23.0.0 in \ linux*) \ system_name=Linux \ ;; \ darwin*) \ system_name=Darwin \ ;; \ *) \ system_name=Linux \ ;; \ esac # The extra CMAKE_C_FLAGS are needed because otherwise boringssl fails to # compile in release mode on some systems with gcc 12 (e.g. Fedora). # In addition, guard these options with -Wno-unknown-warning-option to # prevent clang from failing on them. cmake -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_POSITION_INDEPENDENT_CODE=on \ -DCMAKE_C_FLAGS="-Wno-unknown-warning-option -Wno-stringop-overflow -Wno-array-bounds" \ -DCMAKE_CXX_COMPILER=g++ \ -DCMAKE_C_COMPILER=gcc \ -DCMAKE_SYSTEM_NAME=$system_name \ -DCMAKE_SYSTEM_PROCESSOR=aarch64 \ -GNinja \ .. ninja # Fix the directory structure so that curl can compile against it. # See https://everything.curl.dev/source/build/tls/boringssl mkdir -p lib ln -sf ../crypto/libcrypto.a lib/libcrypto.a ln -sf ../ssl/libssl.a lib/libssl.a cp -Rf ../include . -- The C compiler identification is AppleClang 15.0.0.15000040 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- The CXX compiler identification is AppleClang 15.0.0.15000040 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Perl: /usr/bin/perl (found version "5.30.3") -- The ASM compiler identification is AppleClang -- Found assembler: /Library/Developer/CommandLineTools/usr/bin/gcc -- Configuring done (0.6s) -- Generating done (0.0s) -- Build files have been written to: /Users/me/Documents/mydir/curl-impersonate/build/boringssl/build [189/424] Building C object crypto/CMakeFiles/crypto.dir/x509/t_x509.c.o FAILED: crypto/CMakeFiles/crypto.dir/x509/t_x509.c.o /usr/bin/gcc -DBORINGSSL_IMPLEMENTATION -I/Users/me/Documents/mydir/cli/curl/curl_impersonate_build/curl-impersonate/build/boringssl/third_party/googletest/include -I/Users/me/Documents/mydir/cli/curl/curl_impersonate_build/curl-impersonate/build/boringssl/crypto/../include -Wno-unknown-warning-option -Wno-stringop-overflow -Wno-array-bounds -Werror -Wformat=2 -Wsign-compare -Wmissing-field-initializers -Wwrite-strings -Wvla -Wshadow -ggdb -Wall -fvisibility=hidden -fno-common -Wnewline-eof -fcolor-diagnostics -Wimplicit-fallthrough -Wmissing-prototypes -Wold-style-definition -Wstrict-prototypes -D_XOPEN_SOURCE=700 -O3 -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk -fPIC -MD -MT crypto/CMakeFiles/crypto.dir/x509/t_x509.c.o -MF crypto/CMakeFiles/crypto.dir/x509/t_x509.c.o.d -o crypto/CMakeFiles/crypto.dir/x509/t_x509.c.o -c /Users/me/Documents/mydir/curl-impersonate/build/boringssl/crypto/x509/t_x509.c /Users/me/Documents/mydir/cli/curl/curl_impersonate_build/curl-impersonate/build/boringssl/crypto/x509/t_x509.c:321:18: error: variable 'l' set but not used [-Werror,-Wunused-but-set-variable] int ret = 0, l, i; ^ 1 error generated. [198/424] Building C object crypto/CMakeFiles/crypto.dir/curve25519/curve25519.c.o ninja: build stopped: subcommand failed. gmake: *** [Makefile:268: /Users/me/Documents/mydir/curl-impersonate/build/boringssl/build/lib/libssl.a] Error 1 ```

Key error:

/Users/me/Documents/mydir/cli/curl/curl_impersonate_build/curl-impersonate/build/boringssl/crypto/x509/t_x509.c:321:18: error: variable 'l' set but not used [-Werror,-Wunused-but-set-variable]
    int ret = 0, l, i;
                 ^
1 error generated.
[198/424] Building C object crypto/CMakeFiles/crypto.dir/curve25519/curve25519.c.o
ninja: build stopped: subcommand failed.
gmake: *** [Makefile:268: /Users/me/Documents/mydir/curl-impersonate/build/boringssl/build/lib/libssl.a] Error 1
Explosion-Scratch commented 9 months ago

I also tried brew reinstall nss && brew reinstall ca-certificates, as well as brew reinstall libidn2 (Using force parameters where needed)

lwthiker commented 9 months ago

The binary seems to fail because it is built for x86 and you are on arm64 (mac m1 or m2).

The build from source seems like an issue that needs to be fixed. There are no automatic builds on m1/m2 so these issues are not detected in advance.

Explosion-Scratch commented 9 months ago

Yeah, I figured because of the have 'arm64', need 'x86_64', but Homebrew is showing support for mac Apple Silicon for that dep, so I thought it was likely an issue with this project somehow not selecting the right one

Explosion-Scratch commented 9 months ago
image
dzirtusss commented 7 months ago

FYI #205 a brew receipt you may try

lwthiker commented 3 months ago

With #217 merged, curl-impersonate now builds fine on Mac M2. Moreover, libidn2 id not a dependency for the pre-compiled binaries anymore (a new set of binaries will be released in the upcoming days). So I mark this as closed, please reopen if the errors persist.