dashbitco / broadway_kafka

A Broadway connector for Kafka
233 stars 53 forks source link

Fails to compile on otp 27 #144

Closed GregorGrasselli closed 4 months ago

GregorGrasselli commented 4 months ago

Hi, switching to elixir 1.17.0 and otp 27 breaks this for me.

The setup:

asdf .tool_versions

elixir 1.17.0-otp-27
erlang 27.0

asdf install mix deps.get mix deps.compile

results in:

** (Mix) Could not compile dependency :snappyer, "/home/gregor/.asdf/installs/elixir/1.17.0-otp-27/.mix/elixir/1-17/rebar3 bare compile --paths .../_build/dev/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile snappyer --force", update it with "mix deps.update snappyer" or clean it with "mix deps.clean snappyer"

Obviously, this is not an issue with broadway_kafka directly, but with snappyer, which is a dependency of brod. The reason I'm opening an issue here is that both brod and snappyer are erlang libs and I'm not sure how to reproduce this without mix.

josevalim commented 4 months ago

You can try cloning the projects and running the steps to build them. That should be enough to reproduce the issue.

slashmili commented 4 months ago

I tried on a new elixir app and added only snappyer as dependency.

mix compile
===> Fetching pc v1.14.0
===> Analyzing applications...
===> Compiling pc
===> Compiling _build/default/plugins/pc/src/pc_port_env.erl failed
_build/default/plugins/pc/src/pc_port_env.erl:190:10: code:lib_dir/2 is deprecated; this functionality will be removed in a future release

===> Errors loading plugin pc. Run rebar3 with DEBUG=1 set to see errors.
===> Analyzing applications...
===> Compiling pc
===> Uncaught error in rebar_core. Run with DIAGNOSTIC=1 to see stacktrace or consult rebar3.crashdump
===> When submitting a bug report, please include the output of `rebar3 report "your command"`

We should report it to snappyer.

UPDATE: The error is related to pc


ENV:

Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1]

Interactive Elixir (1.16.1) - press Ctrl+C to exit (type h() ENTER for help)
slashmili commented 4 months ago

I reported the issue on pc https://github.com/blt/port_compiler/issues/82

mhdatie commented 4 months ago

I am experiencing this issue on ELIXIR_VERSION=1.16.2 and OTP_VERSION=26.2.1 when using hexpm/elixir packages. On CircleCI, this is not an issue when using cimg/elixir

slashmili commented 4 months ago

Another related issue reported at https://github.com/zmstone/snappyer/issues/10

slashmili commented 4 months ago

I managed to fix my build by cleaning caches [1]

$ rm -rf  $HOME/.cache/rebar3/hex/hexpm/
$ rm -rf deps _build
$ mix deps.get
$ mix deps.compile

As snappyer doesn't use pin version of pc, I think it uses what you already have in your cache dir

@mhdatie I suggest to delete the cache, maybe it help. Would be good to provide the error message, and specially if it's related to snappyer follow up on https://github.com/zmstone/snappyer/issues/10

deepfryed commented 4 months ago

brod 4.0 removes dependency on snappyer, @josevalim can broadway_kafka be updated to use the latest version of brod ?

josevalim commented 4 months ago

Please send a PR!

deepfryed commented 4 months ago

https://github.com/dashbitco/broadway_kafka/pull/146