haskell-works / hw-kafka-client

Kafka client for Haskell, including auto-rebalancing consumers
MIT License
140 stars 50 forks source link

Callbacks and asynchronous exceptions #177

Open torgeirsh opened 3 years ago

torgeirsh commented 3 years ago

I've made a producer with a delivery callback, and I can print to the console to verify that it's being called. However, if I call Control.Exception.throwTo from the delivery callback (with the main thread id), the next call to produceMessage hangs indefinitely. I can work around it by setting an MVar in the delivery callback instead, and call throwTo from the main thread when it's set, but it feels roundabout and cumbersome. (I know it's not necessary to use throwTo from the main thread, but I wanted to verify that using the exact same thread id works.) Why doesn't throwing from the delivery callback work? Should it work, or have I run into an FFI related snag?

torgeirsh commented 3 years ago

After some investigation, it turns out that this is indeed an FFI snag. I've worked around it by spawning a thread that can throw the exception on behalf of the callback when signaled with an MVar. A better solution would perhaps be to allow throwing exceptions directly in the callback thread. That would require hw-kafka-client to catch any such exceptions when calling callbacks, somehow return them from pollEvents/pollConsumerEvents, and rethrow them "outside" FFI.