Unable to compile it on a MacBook Pro M1

[joeljuca]

I'm having issues compiling erlkaf on a macOS Ventura 13.6.1:

$ mix deps.compile erlkaf
=ERROR REPORT==== 28-Feb-2024::17:31:13.767352 ===
beam/beam_load.c(190): Error loading module rebar3_hex_owner:
  This BEAM file was compiled for a later version of the runtime system than the current (Erlang/OTP 25).
  To fix this, please re-compile this module with an Erlang/OTP 25 compiler.
  (Use of opcode 182; this emulator supports only up to 180.)

=ERROR REPORT==== 28-Feb-2024::17:31:13.767370 ===
Loading of $HOME/.cache/rebar3/plugins/rebar3_hex/ebin/rebar3_hex_owner.beam failed: badfile

===> Errors loading plugin {rebar_cmd,"0.2.6"}. Run rebar3 with DEBUG=1 set to see errors.
librdkafka fork already exist. delete _build/deps/librdkafka for a fresh checkout ...
concurrentqueue fork already exist. delete _build/deps/concurrentqueue for a fresh checkout ...
 LD     erlkaf_nif.so
ld: Undefined symbols:
  _SSL_get_peer_certificate, referenced from:
      _rd_kafka_transport_ssl_handshake in librdkafka.a[70](rdkafka_ssl.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [_build/dev/lib/erlkaf/priv/erlkaf_nif.so] Error 1
make: *** [compile_nif] Error 2
===> Hook for compile failed!

** (Mix) Could not compile dependency :erlkaf, "$HOME/.asdf/installs/elixir/1.15.4-otp-25/.mix/elixir/1-15/rebar3 bare compile --paths _build/dev/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile erlkaf --force", update it with "mix deps.update erlkaf" or clean it with "mix deps.clean erlkaf"

I'm compiling it through Elixir's Mix. It breaks and I don't have much of a clue on how to tackle this. Any help is highly appreciated.

Versions:

• Erlang 25.3.2.3 (asdf)
• Elixir 1.15.4-otp-25 (asdf)
• erlkaf 2.1.4

[silviucpp]

I'm using myself same OS but my mac is MacBook Pro from 2019 and I have no issue.

Some other user reported a similar openssl issue in the past but I couldn't figure out the issue as the user stopped responding: https://github.com/silviucpp/erlkaf/issues/57.

Maybe updating to last librdkafka and switch to openssl 3.0 instead 1.1 might fix your problem.

[joeljuca]

@silviucpp good to hear that you use macOS too.

I read most of #57 before opening this one, I believe the problem is quite similar, but not sure it's the same. I'm back to it this morning, and after deleting my global rebar3 config and trying to rebuild it, I got the following info:

$ DEBUG=1 mix deps.compile erlkaf
===> Expanded command sequence to be run: []
===> Running provider: do
===> Expanded command sequence to be run: [app_discovery,{bare,compile}]
===> Running provider: app_discovery
===> Found top-level apps: [erlkaf]
        using config: [{src_dirs,["src"]},{lib_dirs,["apps/*","lib/*","."]}]
===> Running provider: {bare,compile}
===> Compile (untagged)
===> Running hooks for compile in app erlkaf (./deps/erlkaf) with configuration:
===>    {pre_hooks, [{"(linux|darwin)",compile,"make compile_nif"}]}.
librdkafka fork already exist. delete _build/deps/librdkafka for a fresh checkout ...
concurrentqueue fork already exist. delete _build/deps/concurrentqueue for a fresh checkout ...
 LD     erlkaf_nif.so
ld: Undefined symbols:
  _SSL_get_peer_certificate, referenced from:
      _rd_kafka_transport_ssl_handshake in librdkafka.a[70](rdkafka_ssl.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [./_build/dev/lib/erlkaf/priv/erlkaf_nif.so] Error 1
make: *** [compile_nif] Error 2
===> Hook for compile failed!

** (Mix) Could not compile dependency :erlkaf, "$HOME/.asdf/installs/elixir/1.15.4-otp-25/.mix/elixir/1-15/rebar3 bare compile --paths ./_build/dev/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile erlkaf --force", update it with "mix deps.update erlkaf" or clean it with "mix deps.clean erlkaf"

I see this Undefined symbols error at line 15 and I'm wondering: could it be that concurrentqueue is being built with an inappropriate C compiler? I'm not familiar with C/C++ tooling, but if this error happens in macOS only, it might be some incompatibility with stock macOS compilers.

---

FYI, my previous rebar3.config had nothing more than two plugins in it:

{plugins, [{rebar3_hex, "7.0.7"}, {rebar3_lfe, "0.4.0"}]}.

I added rebar3_hex when I had the first error, but still I had no luck finishing compilation.

[silviucpp]

The undefined symbol is part of openssl _SSL_get_peer_certificate. Has nothing to do with concurrentqueue. As the error says as well: _rd_kafka_transport_ssl_handshake from librdkafka it's using that function.

I have a feling that for some reason the compiler is trying to use the openssl 3.x not 1.1