search

lwthiker / curl-impersonate

curl-impersonate: A special build of curl that can impersonate Chrome & Firefox
MIT License
3.45k stars 229 forks source link

Unable to build firefox or chrome due to SSL interpreting warnings as errors. #201

Closed SuspiciousDuck closed 3 months ago

SuspiciousDuck commented 8 months ago

I am using Arch and was attempting to build libcurl-impersonate from source. However, the both the firefox and chrome AUR packages keep failing to build, even cloning the repo and building it manually doesn't build right. Oddly enough, its the SSL packages that keep failing to build. The error message I get when building firefox is:

FAILED: obj/lib/ssl/ssl.ssl3exthandle.o
cc -MMD -MF obj/lib/ssl/ssl.ssl3exthandle.o.d -DNSS_FIPS_DISABLED -DNSS_NO_INIT_SUPPORT -DNSS_X86_OR_X64 -DNSS_X64 -DNSS_USE_64 -DNSS_ALLOW_SSLKEYLOGFILE -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -DLINUX2_1 -DLINUX -Dlinux -D_DEFAULT_SOURCE -D_BSD_SOURCE -D_POSIX_SOURCE -DSDB_MEASURE_USE_TEMP_DIR -DHAVE_STRERROR -DXP_UNIX -D_REENTRANT -DNSS_DISABLE_DBM -DNSS_DISABLE_LIBPKIX -DNDEBUG -I/home/xxxx/.cache/yay/curl-impersonate-firefox/src/curl-impersonate-0.5.4/build/nss-3.87/dist/Release/include/nspr -I/home/xxxx/.cache/yay/curl-impersonate-firefox/src/curl-impersonate-0.5.4/build/nss-3.87/dist/private/nss -I/home/xxxx/.cache/yay/curl-impersonate-firefox/src/curl-impersonate-0.5.4/build/nss-3.87/dist/public/nss -fPIC -pipe -ffunction-sections -fdata-sections -m64 -Werror -Wall -Wshadow -O2 -std=c99  -c ../../lib/ssl/ssl3exthandle.c -o obj/lib/ssl/ssl.ssl3exthandle.o
../../lib/ssl/ssl3exthandle.c:205:1: error: conflicting types for ‘ssl3_ClientSendSessionTicketXtn’ due to enum/integer mismatch; have ‘PRInt32(const sslSocket *, TLSExtensionData *, sslBuffer *, PRBool *)’ {aka ‘int(const struct sslSocketStr *, struct TLSExtensionDataStr *, struct sslBufferStr *, int *)’} [-Werror=enum-int-mismatch]
  205 | ssl3_ClientSendSessionTicketXtn(const sslSocket *ss, TLSExtensionData *xtnData,
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../lib/ssl/ssl3exthandle.c:17:
../../lib/ssl/ssl3exthandle.h:116:11: note: previous declaration of ‘ssl3_ClientSendSessionTicketXtn’ with type ‘SECStatus(const sslSocket *, TLSExtensionData *, sslBuffer *, PRBool *)’ {aka ‘enum _SECStatus(const struct sslSocketStr *, struct TLSExtensionDataStr *, struct sslBufferStr *, int *)’}
  116 | SECStatus ssl3_ClientSendSessionTicketXtn(const sslSocket *ss,
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

and the error message when building chrome is:

FAILED: ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o
/usr/bin/g++ -DBORINGSSL_IMPLEMENTATION -I/home/xxxx/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.5.4/build/boringssl/third_party/googletest/include -I/home/xxxx/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.5.4/build/boringssl/ssl/test/../../include -Werror -Wformat=2 -Wsign-compare -Wmissing-field-initializers -Wwrite-strings -Wvla -Wshadow -ggdb -Wall -fvisibility=hidden -fno-common -Wno-free-nonheap-object -Wimplicit-fallthrough -Wmissing-declarations -std=c++11 -fno-exceptions -fno-rtti -O3 -DNDEBUG -fPIE -MD -MT ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o -MF ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o.d -o ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o -c /home/xxxx/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.5.4/build/boringssl/ssl/test/settings_writer.cc
/home/xxxx/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.5.4/build/boringssl/ssl/test/settings_writer.cc: In member function ‘bool SettingsWriter::Commit()’:
/home/xxxx/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.5.4/build/boringssl/ssl/test/settings_writer.cc:78:61: error: ignoring attributes on template argument ‘int (*)(FILE*)’ [-Werror=ignored-attributes]
   78 |   using ScopedFILE = std::unique_ptr<FILE, decltype(&fclose)>;
      |                                                             ^
cc1plus: all warnings being treated as errors

Is there a way to fix this? I can't seem to get a working command to replace the flags with sed, but I hacked together a command to find matches using The Silver Searcher.

$ ag ' -Wall| -Werror| -Wno-error' -l ~/curl-impersonate/
~/curl-impersonate/tests/Dockerfile
~/curl-impersonate/build/nss-3.87/nss/doc/rst/legacy/nss_releases/nss_3.55_release_notes/index.rst
~/curl-impersonate/build/nss-3.87/nss/doc/rst/legacy/nss_3.12_release_notes.html/index.rst
~/curl-impersonate/build/nss-3.87/nss/coreconf/NetBSD.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/SunOS4.1.3_U1.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/ReliantUNIX.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/NCR3.0.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/OpenBSD.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/BSD_OS.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/OS2.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/QNX.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/FreeBSD.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/Werror.mk
~/curl-impersonate/build/nss-3.87/nss/coreconf/SunOS5.mk
~/curl-impersonate/build/nss-3.87/nss/gtests/common/gtest.mk
~/curl-impersonate/build/nss-3.87/nss/gtests/google_test/gtest/cmake/internal_utils.cmake
~/curl-impersonate/build/nss-3.87/nss/lib/freebl/mpi/README
~/curl-impersonate/build/nss-3.87/nspr/configure.in
~/curl-impersonate/build/nss-3.87/nspr/configure
~/curl-impersonate/build/nss-3.87/nspr/pr/src/misc/dtoa.c
~/curl-impersonate/build/boringssl/CMakeLists.txt
~/curl-impersonate/build/boringssl/third_party/googletest/cmake/internal_utils.cmake
~/curl-impersonate/build/brotli-1.0.9/c/common/dictionary.c
SuspiciousDuck commented 8 months ago

Update: I was able to get Chrome compiling and working correctly after removing all appearances of -Wall and -Werror from /curl-impersonate/build/boringssl/CMakeLists.txt, but I haven't tried doing the same with Firefox because I don't want to do all of that manually.

$ curl_chrome116 --version
curl 8.1.1 (x86_64-pc-linux-gnu) libcurl/8.1.1 BoringSSL zlib/1.3 brotli/1.0.9 zstd/1.5.5 libidn2/2.3.4 libpsl/0.21.2 (+libidn2/2.3.4) nghttp2/1.56.0
Release-Date: 2023-05-23
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB PSL SSL threadsafe UnixSockets zstd
SuspiciousDuck commented 8 months ago

@lwthiker Possible fix could be cloning the repositories of the SSL libraries, replacing all occurrences of -Wall and -Werror with a blank string, then building.

lwthiker commented 4 months ago

@SuspiciousDuck May you try out https://github.com/lwthiker/curl-impersonate/pull/217? I upgraded the BoringSSL version to the latest stable one, and confirmed that it compiles on Arch without any error. Would be great if you could give it a try as well.

SuspiciousDuck commented 3 months ago

@lwthiker Can I see how you tried doing it? Mine didn't turn out so well. output

SuspiciousDuck commented 3 months ago

What I noticed is that at the bottom, you can see that it is clearly just some malformed curl output. Specifically I noticed that they happened AFTER all of the unzip and tar errors. This means that the build instructions aren't waiting for the downloads. Aside from the download errors, that means I can't test to see if the errors are resolved.

edit: I think I just cracked the case. I went to clone the repo while looking at the PKGBUILD for reference, and while it was trying to compile, I had an idea. I noticed that a lot of the text was overlapping from multiple different lines. This was when I realized I had -j12 in the make process. That was why it wasn't waiting for the download. tldr: -j12 ruins everything

SuspiciousDuck commented 3 months ago

That being said, now that I can actually test building curl-impersonate-chrome, I now see that no, there is still one more warning in BorringSSL, which errors the build. I think a possible solution is to just make another patch. You download the same archives, so why not just patch those archives to fix the build warnings? This was the only warning I got.

[388/452] Building CXX object ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o
FAILED: ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o 
/usr/bin/g++ -DBORINGSSL_IMPLEMENTATION -I/home/user/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.6.0/build/boringssl/third_party/googletest/include -I/home/user/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.6.0/build/boringssl/ssl/test/../../include -Werror -Wformat=2 -Wsign-compare -Wmissing-field-initializers -Wwrite-strings -Wvla -Wshadow -ggdb -Wall -fvisibility=hidden -fno-common -Wno-free-nonheap-object -Wimplicit-fallthrough -Wmissing-declarations -std=c++11 -fno-exceptions -fno-rtti -O3 -DNDEBUG -fPIE -MD -MT ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o -MF ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o.d -o ssl/test/CMakeFiles/bssl_shim.dir/settings_writer.cc.o -c /home/user/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.6.0/build/boringssl/ssl/test/settings_writer.cc
/home/user/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.6.0/build/boringssl/ssl/test/settings_writer.cc: In member function ‘bool SettingsWriter::Commit()’:
/home/user/.cache/yay/curl-impersonate-chrome/src/curl-impersonate-0.6.0/build/boringssl/ssl/test/settings_writer.cc:78:61: error: ignoring attributes on template argument ‘int (*)(FILE*)’ [-Werror=ignored-attributes]
   78 |   using ScopedFILE = std::unique_ptr<FILE, decltype(&fclose)>;
      |                                                             ^
cc1plus: all warnings being treated as errors

edit: I tried compiling curl-impersonate-firefox, and that one actually compiled!! and installed!!

lwthiker commented 3 months ago

tldr: -j12 ruins everything

Yes, parallel build doesn't work, it's a known issue which I haven't been able to solve yet.

That being said, now that I can actually test building curl-impersonate-chrome, I now see that no, there is still one more warning in BorringSSL, which errors the build.

This error was fixed in BoringSSL (https://boringssl.googlesource.com/boringssl/+/e4f60679caa293c047be69f57fc48b46c7452327). Try rebuilding: check out the branch #217, delete the build directory and run make chrome-build again.

SuspiciousDuck commented 3 months ago

Thanks a lot! Issue is resolved.