search

liftbridge-io / liftbridge

Lightweight, fault-tolerant message streams.
https://liftbridge.io
Apache License 2.0
2.57k stars 107 forks source link

Resuming a paused stream/partition using an API or on subscription #250

Closed Jmgr closed 4 years ago

Jmgr commented 4 years ago

Liftbridge offers the option of pausing a stream (with some or all of its partitions) to reduce the amount of CPU and memory usage on idle streams. When that happens, every subscriber to a partition receives an error. Publishing to a paused partition will resume it, allowing it to function as before.

Our use-case includes a situation where we would like to read from a partition even if it is paused. It has to be resumed to do so, but we do not want to publish to it. We have found two possible options to do so: either provide an API to resume a stream/partition, or add a boolean to the subscription request to allow it to resume a partition instead of returning an error.

We are ready to create a PR with a working implementation of either options, but would like to know your opinion on this, @tylertreat.

tylertreat commented 4 years ago

I like the boolean option on subscribe. Let's go with that.

Jmgr commented 4 years ago

PRs created: https://github.com/liftbridge-io/liftbridge-api/pull/34 https://github.com/liftbridge-io/liftbridge/pull/255 https://github.com/liftbridge-io/go-liftbridge/pull/75

Jmgr commented 4 years ago

All PRs have been merged, so I think this can be closed.