search

Azure / azure-documentdb-changefeedprocessor-dotnet

This library provides a host for distributing change feed events in partitioned collection across multiple observers. Instances of the host can scale up (by adding) or down (by removing) dynamically, and the load will be automatically distributed among active instances in about-equal way.
Other
51 stars 22 forks source link

DOC's update on using cosmosdb as a message store/topic/log #135

Open yahorsi opened 5 years ago

yahorsi commented 5 years ago

Once CosmosDb and change feed now is advertized as sor ot messaging/event sourcing store solution there are several questions that must be fully covered in the docs:

  1. What API's support change feed
  2. End to end delivery guarantees. Like what happens if processor dies, or losts network connection
  3. What happens when another processor steals partition lease, is that possible that severeal messages will be delivered to the old and new processor. Is that possible that both processors will be processing same messages simultaniously and so on
  4. Duplicates detection. Like imagine we add new doc to the CosmosDb and got a network problem after message was successfully sent to the server. Client will have to retry the request that might cause duplicates. In RDBMS this is handled by transactions, how is that handled in CosmosDb

Ideally using cosmosdb as a message store/topic/log should have separate coverage in the docs with mentioning all typical messaging topics.

bartelink commented 5 years ago

I second this (and should probably bring over some points from equivalent issues I've raised.)

regarding point 0 - I feel the underlying APIs are an implementation detail that should not be part of the docs here, or at least a separated part of the readme. (or can you clarify your ask?)

Regarding point 3, I feel you should remove this and/or at least reword it - this lib and repo is all about processing the ChangeFeed; The answer to your question is that you can use etags and make the writes contingent on that and/or have a stored proc do some cross document transactions [within the same logical partition]

ealsur commented 5 years ago

I agree, our docs are pending details and we will address it. I have started work on the Azure Functions Change Feed implementation docs, and will keep working on it to also include Change Feed Processor.