search

haskell-works / hw-kafka-client

Kafka client for Haskell, including auto-rebalancing consumers
MIT License
139 stars 50 forks source link

Fix shutdown crash when log/error callback is set #169

Closed phile314 closed 3 years ago

phile314 commented 3 years ago

The rd_kafka_destroy method may call back into Haskell to perform some logging. This is not allowed when attaching the finalizer using Foreign.ForeignPtr. Hence we switch to Foreign.Concurrent, which allows RdKafka to call back into Haskell. We also disable closing the consumer automatically in the finalizer, this should be done by the application by calling closeConsumer explicitly.

phile314 commented 3 years ago

Using our test-suite I could reliably reproduce the error: a C finalizer called back into Haskell. crash using master. With this diff, everything works as it should and the application terminates properly.

phile314 commented 3 years ago

The CI errors seems to be because it uses a librdkafka version from March 2018, but rd_kafka_destroy_flags has been added in August 2018. From the git log, it seems version 0.11.6 (revision 849c066b559950b02e37a69256f0cb7b04381d0e) from October 2018 would suffice.

AlexeyRaga commented 3 years ago

Thanks a lot! It was a long standing issue...