search

pyeventsourcing / esdbclient

Python gRPC client for EventStoreDB
BSD 3-Clause "New" or "Revised" License
46 stars 10 forks source link

Can we trust the event store as a message broker? #16

Closed ikasymov closed 8 months ago

ikasymov commented 11 months ago

I'm using event store consumers in our system as a data source for the read model. How much can I trust the event store as a message broker? That some events will reach listeners with a guarantee without loss.

ikasymov commented 11 months ago

the consumer crashes periodically with this error, once every 2-3 days, you have to restart it.

status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:2113 {created_time:"2023-12-04T15:49:13.388584148+00:00", grpc_status:14, grpc_message:"Socket closed"}"
johnbywater commented 11 months ago

I've asked the EventStoreDB team for answers to your questions. Hopefully we'll get you some answers soon.

johnbywater commented 11 months ago

Hey @ikasymov!

The answers to your questions are:

  1. Messages are guaranteed to be delivered. If by "consumer" you are referring to consuming from a persistent subscription, I've been told to tell you "don't use a persistent subscription for a read model, it creates more problems than it solves." Instead use a catch-up subscription and record the position of the event you are processing atomically with updates to the read model (and use the last position when resuming).

  2. We don't know why the socket is getting closed. There's a "max age" for a gRPC connection but I still don't know what this age is. But I've been told to suggest that reconnecting after 2-3 days isn't "that big of a deal, just detect it and reconnect".

What do you think? Let me know if you have any further thoughts about this?

alexeyzimarev commented 10 months ago

Btw, it's better to ask questions about ESDB in https://discuss.eventstore.com

johnbywater commented 10 months ago

Thanks @alexeyzimarev for the direction.