search

haskell-works / hw-kafka-client

Kafka client for Haskell, including auto-rebalancing consumers
MIT License
140 stars 50 forks source link

librdkafka uses thread-local storage which requires forkOS #155

Closed andrewthad closed 4 years ago

andrewthad commented 4 years ago

I don't use this library, but I just ran into an issue with thread-local storage in a kafka client that I rolled by hand in a Haskell application. librdkafka doesn't do a great job drawing attention to this, but certain functions must run on dedicated OS threads. If you try to run a producer and consumer in the same application (using forkIO for concurrency), it will crash. This library should either document that or do some shenanigans with forkOS to make sure the user doesn't mess this up.

andrewthad commented 4 years ago

Nevermind, this is totally wrong. I misunderstood how librdkafka worked.