search

mtesseract / nakadi-client

Haskell Client Library for the Nakadi Event Broker
Other
13 stars 9 forks source link

Event source #113

Closed mtesseract closed 6 years ago

mtesseract commented 6 years ago

Implement simply event sourcing.

Up for discussion, probably requires some improvements.

This PR also removes the weirdness of the previous subscriptionProcess API where a ConsumeParameters value had to be provided. This is now implicit in the main configuration.

mtesseract commented 6 years ago

@etorreborre I am still thinking about some aspects here, but this might be a start.

mtesseract commented 6 years ago

Hi,

just wanna add that I thought again about the semantics of the streamLimit setting in the context of subscriptionSource and I think it is reasonable as I it is right now in the PR:

What the PR basically does is shifting all responsibility for the streamLimit handling over to the Nakadi server. There's always the problem of Nakadi reconnects during streaming and doing the book keeping on the client side such that exactly streamLimit (distinct!) events are produced by the source is rather tricky.

As a side note, maybe, at some point, Nakadi would also implement a behaviour such that there are no reconnects if you are doing finite streaming (i.e., if streamLimit is specified).

All that being said, I think we should not put too much effort into these questions. What this PR tries to implement is a simple and deterministic behaviour where most actual work is delegated to the Nakadi server.

Feel free to do the review!

etorreborre commented 6 years ago

That looks good to me, I will try this version with our nakadi-cli tool.