Closed jonsterling closed 9 years ago
@larskuhtz When you have a moment, would you mind taking a look at this pull request? It may be best to look mostly at the final diff, since things changed a bit along the way and each individual commit is perhaps not quite so informative.
The main changes here are as follows:
PutRecords
requests) was pretty bizarre and wrong; I don't think that it would have caused any bugs, but it probably didn't behave quite how I had hoped. I have replaced it with code that seems a lot more straightforward and actually appears to work properly (see takeTBMQueueWithTimeout
and chunkSource
). Perhaps there are still bugs in it which I have not observed, though.MonadError
; this caused a lot of bureaucracy and made things a lot more complicated & ornate than they needed to be. I've made the thing internally use normal exceptions, but exposed APIs returning a suitable error type.withKinesisProducer
exits, then the remaining messages are flushed out with a configurable timeout.MonadBaseControl IO
, etc., but I found that no expressivity had been gained over just using IO
directly. So a number of things have been simplified in that respect, though the public APIs are all polymorphic over their monad.Some things to look for in particular:
I'll take a look. I'll probably need some time to read through all of it.
Previously pretty much everything was plumbed into MonadError; this caused a lot of bureaucracy and made things a lot more complicated & ornate than they needed to be. I've made the thing internally use normal exceptions, but exposed APIs returning a suitable error type.
While working on similar issues in yet-another-logger I just made a few experiences that make me wonder if the apparent complexity of MonadError
and the apparent simplicity of normal Exceptions is somewhat spurious. Maybe with MonadError
certain edge cases are just explicit and one is forced to deal with them, while with normal exceptions it's easy to overlook certain subtleties in the semantics of throw and catch. The more I dig in the exceptions in a multithreaded setup the more I wonder if I really understand how they work.
I am going to merge this now. To summarize:
TQueue
which causes live-locking under heavy load; the bug was a strict reverse of the internal queue, which should have been lazy. The Producer now ships by default using a queue based on TBMChan
, but it may be configured to use TBMQueue
(or any other implementation), which user may wish to do once STM is patched, since TQueue
has better throughput in general than TChan
.m (Either SpecificErrorType a)
.
This is intended to address #19, #17, #21, and #22.
There is more to be done before this may be merged.
MonadError
either
dependencym (Either e a)
enclosed-exceptions
where appropriatePutRecord
endpoint