search

jasonacox / Build-OpenSSL-cURL

Scripts to build OpenSSL, HTTP/2 (nghttp2) and cURL (libcurl) for MacOS, iOS and tvOS devices (x86_64, armv7, armv7s, arm64, arm64e). Now Supporting Apple Silicon, OpenSSL 3.0.x with TLS 1.3 and Mac Catalyst builds.
MIT License
419 stars 125 forks source link

build issue #49

Closed jeffreyc2017 closed 2 years ago

jeffreyc2017 commented 3 years ago

When I built this script on mac Big Sur I got following error:

% git clone git@github.com:jasonacox/Build-OpenSSL-cURL.git
% cd Build-OpenSSL-cURL
% ./build.sh 

...

Building Mac libraries
Building curl-7.72.0 for x86_64
  Copying headers
Building iOS libraries (bitcode)
Building curl-7.72.0 for iPhoneOS  armv7 bitcode
** ERROR with Build - Check /tmp/curl*.log
==> /tmp/curl-7.72.0-iOS-armv7-bitcode.log <==
make[2]: *** [curl] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

==> /tmp/curl-7.72.0-x86_64.log <==
test -z "projects/Windows/VC6/lib/libcurl.dsp.dist projects/Windows/VC6/src/curl.dsp.dist projects/Windows/VC7/lib/libcurl.vcproj.dist projects/Windows/VC7/src/curl.vcproj.dist projects/Windows/VC7.1/lib/libcurl.vcproj.dist projects/Windows/VC7.1/src/curl.vcproj.dist projects/Windows/VC8/lib/libcurl.vcproj.dist projects/Windows/VC8/src/curl.vcproj.dist projects/Windows/VC9/lib/libcurl.vcproj.dist projects/Windows/VC9/src/curl.vcproj.dist projects/Windows/VC10/lib/libcurl.vcxproj.dist projects/Windows/VC10/src/curl.vcxproj.dist projects/Windows/VC11/lib/libcurl.vcxproj.dist projects/Windows/VC11/src/curl.vcxproj.dist projects/Windows/VC12/lib/libcurl.vcxproj.dist projects/Windows/VC12/src/curl.vcxproj.dist projects/Windows/VC14/lib/libcurl.vcxproj.dist projects/Windows/VC14/src/curl.vcxproj.dist projects/Windows/VC15/lib/libcurl.vcxproj.dist projects/Windows/VC15/src/curl.vcxproj.dist" || rm -f projects/Windows/VC6/lib/libcurl.dsp.dist projects/Windows/VC6/src/curl.dsp.dist projects/Windows/VC7/lib/libcurl.vcproj.dist projects/Windows/VC7/src/curl.vcproj.dist projects/Windows/VC7.1/lib/libcurl.vcproj.dist projects/Windows/VC7.1/src/curl.vcproj.dist projects/Windows/VC8/lib/libcurl.vcproj.dist projects/Windows/VC8/src/curl.vcproj.dist projects/Windows/VC9/lib/libcurl.vcproj.dist projects/Windows/VC9/src/curl.vcproj.dist projects/Windows/VC10/lib/libcurl.vcxproj.dist projects/Windows/VC10/src/curl.vcxproj.dist projects/Windows/VC11/lib/libcurl.vcxproj.dist projects/Windows/VC11/src/curl.vcxproj.dist projects/Windows/VC12/lib/libcurl.vcxproj.dist projects/Windows/VC12/src/curl.vcxproj.dist projects/Windows/VC14/lib/libcurl.vcxproj.dist projects/Windows/VC14/src/curl.vcxproj.dist projects/Windows/VC15/lib/libcurl.vcxproj.dist projects/Windows/VC15/src/curl.vcxproj.dist
rm -rf .libs _libs
rm -f *.lo

...

% less /tmp/curl-7.72.0-iOS-armv7-bitcode.log

...

ld: warning: ignoring file /usr/local/lib/libz.dylib, building for iOS-armv7 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture armv7:
  "_inflate", referenced from:
      _hugehelp in curl-tool_hugehelp.o
      _inflate_stream in libcurl.a(libcurl_la-content_encoding.o)
     (maybe you meant: _nghttp2_session_get_hd_inflate_dynamic_table_size, _nghttp2_hd_inflate_get_max_dynamic_table_size , _nghttp2_hd_inflate_get_table_entry , _nghttp2_hd_inflate_get_num_table_entries , _nghttp2_hd_inflate_del , _nghttp2_hd_inflate_new2 , _nghttp2_hd_inflate_hd , _nghttp2_hd_inflate_new , _nghttp2_hd_inflate_init , _nghttp2_hd_inflate_free , _nghttp2_hd_inflate_get_dynamic_table_size , _nghttp2_hd_inflate_hd_nv , _nghttp2_hd_inflate_change_table_size , _nghttp2_hd_inflate_end_headers , _nghttp2_hd_inflate_hd2 )
  "_inflateEnd", referenced from:
      _hugehelp in curl-tool_hugehelp.o
      _inflate_stream in libcurl.a(libcurl_la-content_encoding.o)
      _exit_zlib in libcurl.a(libcurl_la-content_encoding.o)
  "_inflateInit2_", referenced from:
      _hugehelp in curl-tool_hugehelp.o
      _inflate_stream in libcurl.a(libcurl_la-content_encoding.o)
      _gzip_init_writer in libcurl.a(libcurl_la-content_encoding.o)
  "_inflateInit_", referenced from:
      _deflate_init_writer in libcurl.a(libcurl_la-content_encoding.o)
  "_zlibVersion", referenced from:
      _curl_version in libcurl.a(libcurl_la-version.o)
      _curl_version_info in libcurl.a(libcurl_la-version.o)
      _gzip_init_writer in libcurl.a(libcurl_la-content_encoding.o)
ld: symbol(s) not found for architecture armv7
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [curl] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1
jasonacox commented 3 years ago

hi @jeffreyc2017 -

building for iOS-armv7 but attempting to link with file built for macOS-x86_64

Did you happen to install the Xcode update and iOS SDKs? It is trying to link in the Apple libz but not finding the arm7 iOS architecture on your local system. This may be an issue with Big Sur. I need to upgrade my intel gear to test. I have Big Sur compiling arm7 targets on Apple Silicon but not on intel yet.

jeffreyc2017 commented 3 years ago

Thanks @jasonacox

Yeah I've updated Xcode to 12.2 and iPhoneOS14.2.sdk exists in /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs. My MBP is using Intel Core i7 with macOS Big Sur 11.0.1.

% xcodebuild -version
Xcode 12.2
Build version 12B45b

% system_profiler SPDeveloperToolsDataType
Developer:

    Developer Tools:

      Version: 12.2 (12B45b)
      Location: /Applications/Xcode.app
      Applications:
          Xcode: 12.2 (17535)
          Instruments: 12.2 (64541.11)
      SDKs:
          iOS:
              14.2: (18B79)
          iOS Simulator:
              14.2: (18B79)
          macOS:
              11.0: (20A2408)
              20.0: 
          tvOS:
              14.2: (18K54)
          tvOS Simulator:
              14.2: (18K54)
          watchOS:
              7.1: (18R579)
          watchOS Simulator:
              7.1: (18R579)

% which make
/usr/bin/make

% make -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
jasonacox commented 3 years ago

Hi @jeffreyc2017 - I'm not able to reproduce your issue with Big Sur. I'm able to get the binaries built and linked without error.

I did notice this line:

ld: warning: ignoring file /usr/local/lib/libz.dylib, building for iOS-armv7 but attempting to link with file built for macOS-x86_64

That (/usr/local path) seems to indicate that you have installed a custom (non-Apple) version of libz.dylib. That would likely conflict with linking to the correct libraries for the different architectures that are located here: /Applications/Xcode.app/Contents/Developer/Platforms/

I recommend you try removing the custom built libz and use the one provided by Xcode to see if that fixes the problem.

jasonacox commented 3 years ago

Hi @jeffreyc2017 - Let me know if that helped or if you still see the same issue with the latest build script.

jeffreyc2017 commented 3 years ago

Thanks @jasonacox . Yeah I'm still getting the same issue with the latest build script. Still no idea how to fix it.

jasonacox commented 3 years ago

Ok. I recommend disabling the non-native /usr/local/lib/libz.dylib that you have in your build include path and force xcode to use the Apple library. You can try to temporarily disable it with something like:

sudo mv /usr/local/lib/libz.dylib /usr/local/lib/libz.dylib-disable
# revert it back with sudo mv /usr/local/lib/libz.dylib-disable /usr/local/lib/libz.dylib

Try to run the build again and see if that helps with the errors.

jeffreyc2017 commented 3 years ago

Thanks @jasonacox . I ran the above command, and also tried removing the libz files in /usr/local/lib to trash bin, still got the error

Building Mac libraries
Building curl-7.74.0 for x86_64 (MacOS 11.2.3)
Testing binary for x86_64:
curl 7.74.0 (x86_64-apple-darwin) libcurl/7.74.0 OpenSSL/1.1.1i zlib/1.2.11 brotli/1.0.9 zstd/1.4.9 libidn2/2.3.0 nghttp2/1.42.0 librtmp/2.3
Release-Date: 2020-12-09
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets zstd
Building curl-7.74.0 for arm64 (MacOS 11.2.3)
** ERROR with Build - Check /tmp/curl*.log
==> /tmp/curl-7.74.0-arm64.log <==
make[2]: *** [curl] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

==> /tmp/curl-7.74.0-x86_64.log <==
test -z "projects/Windows/VC6/lib/libcurl.dsp.dist projects/Windows/VC6/src/curl.dsp.dist projects/Windows/VC7/lib/libcurl.vcproj.dist projects/Windows/VC7/src/curl.vcproj.dist projects/Windows/VC7.1/lib/libcurl.vcproj.dist projects/Windows/VC7.1/src/curl.vcproj.dist projects/Windows/VC8/lib/libcurl.vcproj.dist projects/Windows/VC8/src/curl.vcproj.dist projects/Windows/VC9/lib/libcurl.vcproj.dist projects/Windows/VC9/src/curl.vcproj.dist projects/Windows/VC10/lib/libcurl.vcxproj.dist projects/Windows/VC10/src/curl.vcxproj.dist projects/Windows/VC11/lib/libcurl.vcxproj.dist projects/Windows/VC11/src/curl.vcxproj.dist projects/Windows/VC12/lib/libcurl.vcxproj.dist projects/Windows/VC12/src/curl.vcxproj.dist projects/Windows/VC14/lib/libcurl.vcxproj.dist projects/Windows/VC14/src/curl.vcxproj.dist projects/Windows/VC15/lib/libcurl.vcxproj.dist projects/Windows/VC15/src/curl.vcxproj.dist" || rm -f projects/Windows/VC6/lib/libcurl.dsp.dist projects/Windows/VC6/src/curl.dsp.dist projects/Windows/VC7/lib/libcurl.vcproj.dist projects/Windows/VC7/src/curl.vcproj.dist projects/Windows/VC7.1/lib/libcurl.vcproj.dist projects/Windows/VC7.1/src/curl.vcproj.dist projects/Windows/VC8/lib/libcurl.vcproj.dist projects/Windows/VC8/src/curl.vcproj.dist projects/Windows/VC9/lib/libcurl.vcproj.dist projects/Windows/VC9/src/curl.vcproj.dist projects/Windows/VC10/lib/libcurl.vcxproj.dist projects/Windows/VC10/src/curl.vcxproj.dist projects/Windows/VC11/lib/libcurl.vcxproj.dist projects/Windows/VC11/src/curl.vcxproj.dist projects/Windows/VC12/lib/libcurl.vcxproj.dist projects/Windows/VC12/src/curl.vcxproj.dist projects/Windows/VC14/lib/libcurl.vcxproj.dist projects/Windows/VC14/src/curl.vcxproj.dist projects/Windows/VC15/lib/libcurl.vcxproj.dist projects/Windows/VC15/src/curl.vcxproj.dist
rm -rf .libs _libs
rm -f *.lo

Libraries...

But the log doesn't exist

% cat /tmp/curl-7.74.0-x86_64.log
cat: /tmp/curl-7.74.0-x86_64.log: No such file or directory
jasonacox commented 3 years ago

Building Mac libraries ... ==> /tmp/curl-7.74.0-arm64.log <== make[2]: [curl] Error 1 make[1]: [all-recursive] Error 1 make: *** [all-recursive] Error 1

I know it is a bit misleading but in your snip above, the failure was on the "arm64" build, the x86_64 build log shows a successful build and even the binary test is successful. The error details would be in /tmp/curl-7.74.0-arm64.log not in the x86_64 one. That make Error indicates it is unable to build for the ARM processor (Apple Silicon M1).

You could edit the curl/libcurl-build.sh script and look for this line and add a command below it to display the arm64 logs right after the build to help troubleshoot further:

buildMac "arm64"
# add the following line
cat /tmp/curl-7.74.0-arm64.log

Technically the build script should trap the error and exit immediately but it seems like it is continuing to the next phase which could also be deleting the log files before you have a chance to review them.