dashbitco / broadway_kafka

A Broadway connector for Kafka
232 stars 53 forks source link

the table identifier does not refer to an existing ETS table #114

Closed yordis closed 1 year ago

yordis commented 2 years ago

We got two error reports about the following:

ArgumentError: errors were found at the given arguments:

  * 1st argument: the table identifier does not refer to an existing ETS table

  Module "ets", in :ets.insert/2
  File "lib/broadway_kafka/producer.ex", line 720, in BroadwayKafka.Producer.set_draining_after_revoke!/2
  File "lib/broadway_kafka/producer.ex", line 508, in BroadwayKafka.Producer.assignments_revoked/1
  File "/opt/app/deps/brod/src/brod_group_coordinator.erl", line 502, in :brod_group_coordinator.stabilize/3
  File "/opt/app/deps/brod/src/brod_group_coordinator.erl", line 416, in :brod_group_coordinator.handle_info/2
  File "gen_server.erl", line 695, in :gen_server.try_dispatch/4
  File "gen_server.erl", line 771, in :gen_server.handle_msg/6
  File "proc_lib.erl", line 226, in :proc_lib.init_p_do_apply/3

Not sure who would remove the ETS table. Leaving over here to debug at some point, and gather some public information about what could potentially be the issue.

Environment

ELIXIR_VERSION=1.14.0
OTP_VERSION=24.3.4
ALPINE_VERSION=3.15.3

Deps

{:broadway, "== 1.0.5"},
{:broadway_kafka, "0.4.0"},
slashmili commented 1 year ago

That's odd, it means that the producer process is not alive anymore. The ets table is created during init:

https://github.com/dashbitco/broadway_kafka/blob/50c710a0a29d3f5cb994933a3abf71b7598f9ea9/lib/broadway_kafka/producer.ex#L237-L240

josevalim commented 1 year ago

Can you please include versions for Elixir, OTP, Broadway, Broadway.Kafka and brod?

yordis commented 1 year ago

Can you please include versions for Elixir, OTP, Broadway, Broadway.Kafka and brod?

I updated the description of the ticket, sorry about that

josevalim commented 1 year ago

Is this still happening? Can you provide a way to reproduce it? Thank you.

yordis commented 1 year ago

I do not have access to the code base anymore. Other people could reopen it.