Question

[MaRaSu]

If I'm not mistaken a stream ending in .forEach() returns a Promise that will resolve when all events have been consumed, however a stream ending in .to() will resolve the Promise right in the middle of stream being initialised.

For my use cases I would much prefer also .to() resolving the Promise only after the events have been consumed. Any advise on this?

Example code snippet with Promise resolving too early:

const someStream = kstream.from("input-topic")
    .take(1)
    .to("output-topic")

someStream.then(() => console.log("finished"))

kstream.start();

[krystianity]

Hi @MaRaSu yes you are right, .forEach() promise resolve comes straight from most.js, where as the .to() promise resolve depends on the connection of a potential additional Kafka producer, in some cases this is very essential as the producer needs to be connected before the consumer pipes the first messages through the stream.

Most of our production use-cases however require constant open and running streams anyways, therefore we never implemented such a behaviour. However I might add this in the next release, or at least an option to get this resolved when you call .to().

If you need this very urgently there is a potential workaround for you that we also use for integration tests, you can listen to produce delivery events with the help of an event emitter that you pass to the stream, this way you might be able to await the messages before the suggested feature is merged. We use this here for example.