Homebrew build of proxygen failing

[timsutton]

👋 Hello, proxygen is part of the homebrew-core repository and we've been having issues bumping it since September. Currently it seems like the build error is on building (within samples?) libhttpperf2 for quic::samples::HQLoggerHelper::HQLoggerHelper. Wondered if you have suggestions for how to resolve it?

[ 87%] Linking CXX shared library liblibhttperf2.dylib
cd /tmp/proxygen-20240109-46051-seeuqi/build/proxygen/httpclient/samples/httperf2 && /opt/homebrew/Cellar/cmake/3.28.1/bin/cmake -E cmake_link_script CMakeFiles/libhttperf2.dir/link.txt --verbose=1
/opt/homebrew/Library/Homebrew/shims/mac/super/clang++ -O3 -DNDEBUG -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -dynamiclib -Wl,-headerpad_max_install_names -o liblibhttperf2.dylib -install_name @rpath/liblibhttperf2.dylib CMakeFiles/libhttperf2.dir/HTTPerf2.cpp.o CMakeFiles/libhttperf2.dir/Client.cpp.o  -Wl,-rpath,/tmp/proxygen-20240109-46051-seeuqi/build/proxygen/lib ../../../lib/libproxygen.dylib /opt/homebrew/lib/libwangle.1.0.0.dylib /opt/homebrew/lib/libzstd.dylib /opt/homebrew/lib/libboost_iostreams-mt.dylib -lz /opt/homebrew/lib/libmvfst_fizz_client.a /opt/homebrew/lib/libmvfst_client.a /opt/homebrew/lib/libmvfst_server.a /opt/homebrew/lib/libmvfst_server_state.a /opt/homebrew/lib/libmvfst_transport.a /opt/homebrew/lib/libmvfst_batch_writer.a /opt/homebrew/lib/libmvfst_cc_algo.a /opt/homebrew/lib/libmvfst_codec_pktrebuilder.a /opt/homebrew/lib/libmvfst_happyeyeballs.a /opt/homebrew/lib/libmvfst_async_udp_socket.a /opt/homebrew/lib/libmvfst_events.a /opt/homebrew/lib/libmvfst_looper.a /opt/homebrew/lib/libmvfst_observer.a /opt/homebrew/lib/libmvfst_state_datagram_handler.a /opt/homebrew/lib/libmvfst_state_pacing_functions.a /opt/homebrew/lib/libmvfst_buf_accessor.a /opt/homebrew/lib/libmvfst_state_ack_handler.a /opt/homebrew/lib/libmvfst_loss.a /opt/homebrew/lib/libmvfst_state_simple_frame_functions.a /opt/homebrew/lib/libmvfst_state_functions.a /opt/homebrew/lib/libmvfst_codec_pktbuilder.a /opt/homebrew/lib/libmvfst_state_stream.a /opt/homebrew/lib/libmvfst_flowcontrol.a /opt/homebrew/lib/libmvfst_qlogger.a /opt/homebrew/lib/libmvfst_state_stream_functions.a /opt/homebrew/lib/libmvfst_fizz_handshake.a /opt/homebrew/lib/libmvfst_dsr_frontend.a /opt/homebrew/lib/libmvfst_dsr_types.a /opt/homebrew/lib/libmvfst_transport_knobs.a /opt/homebrew/lib/libmvfst_transport_settings_functions.a /opt/homebrew/lib/libmvfst_state_machine.a /opt/homebrew/lib/libmvfst_codec.a /opt/homebrew/lib/libmvfst_handshake.a /opt/homebrew/lib/libmvfst_codec_packet_number_cipher.a /opt/homebrew/lib/libmvfst_codec_decode.a /opt/homebrew/lib/libmvfst_codec_types.a /opt/homebrew/lib/libmvfst_bufutil.a /opt/homebrew/lib/libmvfst_constants.a /opt/homebrew/lib/libmvfst_exception.a /opt/homebrew/lib/libfizz.1.0.0.dylib /opt/homebrew/lib/libfolly.0.58.0-dev.dylib /opt/homebrew/lib/libboost_context-mt.dylib /opt/homebrew/lib/libboost_filesystem-mt.dylib /opt/homebrew/lib/libboost_atomic-mt.dylib /opt/homebrew/lib/libboost_program_options-mt.dylib /opt/homebrew/lib/libboost_regex-mt.dylib /opt/homebrew/lib/libboost_system-mt.dylib /opt/homebrew/lib/libboost_thread-mt.dylib /opt/homebrew/lib/libglog.dylib /opt/homebrew/lib/libevent.dylib /opt/homebrew/lib/libdouble-conversion.dylib /opt/homebrew/lib/libfmt.10.2.1.dylib /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/lib/libz.tbd /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/lib/libbz2.tbd /opt/homebrew/lib/liblzma.dylib /opt/homebrew/lib/liblz4.dylib /opt/homebrew/lib/libsnappy.dylib /opt/homebrew/lib/libsodium.dylib -lc++abi /opt/homebrew/lib/libssl.dylib /opt/homebrew/lib/libcrypto.dylib /opt/homebrew/lib/libsodium.dylib /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk/usr/lib/libz.tbd /opt/homebrew/lib/libzstd.dylib /opt/homebrew/lib/libgflags.2.2.2.dylib 
ld: Undefined symbols:
  quic::samples::HQLoggerHelper::HQLoggerHelper(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, bool, quic::QuicNodeType), referenced from:
      std::__1::__shared_ptr_emplace<quic::samples::HQLoggerHelper, std::__1::allocator<quic::samples::HQLoggerHelper>>::__shared_ptr_emplace[abi:v160006]<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, bool, quic::QuicNodeType>(std::__1::allocator<quic::samples::HQLoggerHelper>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, bool&&, quic::QuicNodeType&&) in Client.cpp.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [proxygen/httpclient/samples/httperf2/liblibhttperf2.dylib] Error 1
make[1]: *** [proxygen/httpclient/samples/httperf2/CMakeFiles/libhttperf2.dir/all] Error 2

More context and full build logs are over here in the Full install --build-bottle proxygen output.

[timsutton]

To clarify, I can build the libs if I set -DBUILD_SAMPLES=OFF. So it seems the issue is mainly with the sample code?

[lnicco]

thanks for reporting this issue @timsutton. I tried to reproduce locally on MacOS 14.2 but the build completes correctly.

Did you have any luck reproducing locally? That would help us understand the issue and work on a fix

[timsutton]

Just to confirm is your repro on an Apple Silicon machine?

I can repro it locally (on AS) within the same context that a brew build is done, via a brew install --build-from-source proxygen on a branch that bumps these various shared libraries, like this one. I think the basic instructions to recreate it would be (assuming gh CLI is installed):

brew tap homebrew/core
cd $(brew --prefix)/Library/Taps/homebrew/homebrew-core
gh pr checkout 160044
export HOMEBREW_NO_INSTALL_FROM_API=1
# (I might have some of the deps orderings wrong here, this is to ensure they're all built via brew from the same 2024.01.15 revision)
brew install -s mvfst
brew install -s folly fizz wangle
brew install -s proxygen
# ^ should fail, you can add the --debug to give the option to drop into a shell in the proper build env, when it errors

Separately I just tried to see about doing a local build from the repo directly and leaving homebrew out of it, but when I ran build.sh from the proxygen I hit a snag during the Folly build: https://gist.github.com/timsutton/514f2ddc79337dc212f533fb4a690352

If I wanted to try setting up the build differently (as CI does) I could try and follow through the GitHub actions flow here. It looks like the CI uses macOS 12 with Xcode 14.2. Haven't yet tried that though.

[lnicco]

I was able to figure out which commit broke it https://github.com/facebook/proxygen/commit/1a9557d9a481ea60ca7142b15f34528894730f7f

I am however still working on reproducing the build error locally (on a macbook air M2 on Sonoma 14.2.1)

I hit a snag during the Folly build:

Yeah this seems to be an issue with the build.sh script that is somehow picking up the homebrew version (v10) of the fmt library instead of the one that it installs itself (v9). Can you try brew uninstall --ignore-dependencies fmt and rerun build.sh?

[lnicco]

I was able to repro and will land a fix soon.

[lnicco]

please reopen if the patch doesn't fix the issue

[timsutton]

@lnicco Thanks for the fix! Confirmed it fixes the issue, much appreciated!