infinyon / fluvio

Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.
https://www.fluvio.io/
Apache License 2.0
3.88k stars 491 forks source link

3 phases of ACK back to SPU to update current offset. #2369

Closed simlay closed 2 years ago

simlay commented 2 years ago

The last_read_offset (naming not fully determined) we will add to ReplicaSlice needs to be a incremented (and never decremented) via 3 flows:

sehz commented 2 years ago

Client already send ACK back to SPU: https://github.com/infinyon/fluvio/blob/master/crates/fluvio/src/consumer.rs. See UpdateOffsetsRequest

morenol commented 2 years ago

This spawn task do that:

https://github.com/infinyon/fluvio/blob/2d191918329c7f7b97b802022e79deeba1d6a8d1/crates/fluvio/src/consumer.rs#L365

github-actions[bot] commented 2 years ago

Stale issue message