zalky / cues

Queues on cue: low-latency persistent blocking queues, processors, and graphs via Chronicle Queue
Apache License 2.0
105 stars 2 forks source link

At most once #7

Closed zalky closed 1 year ago

zalky commented 1 year ago

This PR

  1. Provides at most once message delivery semantics
  2. Makes a small tweak to unhandled error reporting
  3. Bumps version for new release

Cues provides ::q/exactly-once message delivery by default. With this PR you can now optionally configure graphs to use ::q/at-most-once delivery semantics instead:

(require '[cues.queue :as q])

(defn example-graph
  [db]
  {:id         ::example
   :strategy   ::q/at-most-once
   :processors [{:id ::source}
                ...]})

With at most once semantics any processor step is only ever attempted once, and never retried. While failures may result in dropped messages, this provides two modest benefits if dropped messages are not a problem:

  1. Approximately 30-40% faster graph performance
  2. You can avoid implementing idempotency on side-effects using the delivery hash: processor steps are simply never retried

In contrast at least once semantics pose no meaningful benefits with respect to exactly once delivery, and so outside of processors with side-effects (where it is the default and explained previously) that strategy is not provided.