search

akka / akka-meta

This repository is dedicated to high-level feature discussions and for persisting design decisions.
Apache License 2.0
201 stars 23 forks source link

How shall we surface acknowledged Sinks in Akka Streams? #14

Open rkuhn opened 8 years ago

rkuhn commented 8 years ago

This is the much simpler cousin of #13. The primary proposal is to model this as a Flow:

class KafkaSink extends GraphStage[FlowShape[Message, Confirmation]]

This is fundamentally what happens: messages are sent to Kafka, to be turned into confirmations.

13h3r commented 8 years ago

Originally I emit not Confirmation, but Future[Confirmation]. This allows client to keep order of messages and perform some actions in parallel and then join confirmed record with some other job.

Emitting just Confirmation shuffles original order.

Also Future[Confirmation] can be easily turned into Confirmation and allows client to control how many confirmation may be in progress.

rkuhn commented 8 years ago

Ah, my assumption was that the .mapAsync() is already included in the KafkaSink and that the confirmations are emitted in the right order.

13h3r commented 8 years ago

I wanted to provide as minimal abstraction as possible. Then client will decide do he cares about order (and use mapAsync) or dont (and use mapAsyncUnordered)

Same for #13

rkuhn commented 8 years ago

Yes, this is a good point. This case is rather easy, the Source case is vastly different due to needing three ports.

ktoso commented 8 years ago

I agree with https://github.com/akka/akka-meta/issues/14#issuecomment-195453520 in that we'll want to expose Future[Confirmation], from which can go to Confirmation with or without order :+1: