tenzir / public-roadmap

The public roadmap of Tenzir
https://docs.tenzir.com/roadmap
4 stars 0 forks source link

Async Generators for Operators #99

Closed dominiklohmann closed 8 months ago

dominiklohmann commented 1 year ago

This is a technically motivated roadmap item.

Operators currently have a rather complex scheduling behavior that works around the lack of awaitable coroutines in CAF, Tenzir's underlying actor framework. During development leading up to the Tenzir alpha, we have found the lack of for co_await (auto batch : input) to be actively harmful to developer productivity. Almost all operators that require waiting for some task or request to complete asynchronously currently use difficult-to-understand and brittle workarounds.

To solve this, we want to make caf::result a coroutine type that can await, and implement an async_generator as a basis for operators.

### Definition of Done
- [ ] Implement a coroutine-native version of `caf::result`
- [ ] Implement awaitable messaging primitives for CAF
- [ ] Implement an async generator coroutine type
- [ ] Switch operators to use an async generator
dominiklohmann commented 8 months ago

Closing as this is not something we'll ever schedule; it's simply too much effort. This can only be driven forward by contributing to CAF upstream.