search

rabblerouser / core

Pluggable, extensible membership database for community organising
GNU Affero General Public License v3.0
17 stars 10 forks source link

Don't use a batch size of 1 for the event-forwarder lambdas #131

Closed camjackson closed 6 years ago

camjackson commented 7 years ago

Batch size is currently 1, so that batches can succeed or fail atomically. This prevents double- or zero-handling of events in the case where half a batch succeeded and half failed.

The current solution works, but is a bit of an anti-pattern. More information here: https://github.com/rabblerouser/rabblerouser-event-forwarder#error-handling

Needs further research to determine how a lambda can indicate that some events succeeded and should not be sent again, but other events need to be retried.

tomsulston commented 6 years ago

Batch size of one feels OK for now. Maybe when scaling is an issue we should revisit this.