search

Netflix / conductor-community

Apache License 2.0
61 stars 72 forks source link

nats jetstream support #95

Closed astelmashenko closed 2 years ago

astelmashenko commented 2 years ago

Pull Request type

Changes in this PR

Re-named event-queue/nats into nats-streaming, since nats streaming is deprecated and has jnats 2.6.5 dependency, it does not give implement JetStream support.

event-queue/nats now has nats and jetstream components, latest dependency jnats 2.15.6 is used. Use cases checked:

  1. jetstream is not started before conductor server, it reconnects to jetstream later and working properly
  2. if jetstream component loses connection it is able to self-heal (re-connect)
  3. clustered run of 2 instances connecting and working properly
  4. happy-path of completing Wait tasks and HTTO async complete task.

New properties added:

conductor.default-event-queue.type=jsm
conductor.event-queues.jsm.enabled=true
conductor.event-queues.jsm.pollTimeDuration=1000
conductor.event-queues.jsm.url=nats://localhost:4222
conductor.event-queues.jsm.durableName=durableName
conductor.event-queues.jsm.listenerQueuePrefix=prefix
conductor.event-queues.jsm.streamStorageType=file

JetStream is connected as WorkQueue retention policy and uses JetStream push based queue group subscription, which gives round-robin between subscriptions so messages are not processed several times in clustered env.