feat: add source trait - Githubissues

kaskada-ai / kaskada

Modern, open-source event-processing

https://kaskada.io/

Apache License 2.0

348 stars 15 forks source link

feat: add source trait #803

Closed jordanrfrazier closed 10 months ago

jordanrfrazier commented 11 months ago

Adds the source trait for sources to implement. The idea being they'll be created as async functions and we can just call next on the returned stream -- no need to make them a whole ReadPipeline or anything we think.

Adds the sparrow-sources crate
Adds the sparrow-interfaces crate
Refactors batch into sparrow-batch.

jordanrfrazier commented 11 months ago

TODO: add task for adding machete to ci