https://github.com/chrislusf/gleam -
Gleam is another go based micro-footprint framework used for MR style processing.
There is a parallel thread in Gleam to implement SQL layer on top of it.
It would be better if octosql could have the physical planner running on Gleam
We have a very specified set of qualities we need to achieve with distributed OctoSQL streaming, and I don't think we'll be able to do that with gleam.
What comes to my mind now:
exactly-once stream processing
big on-disk state (for stuff like group by, join, distinct)
working in terms of disks, not nodes (nodes can come and go, disks get attached to them and are scheduled workloads on)
exactly-once stream processing in face of node failures
https://github.com/chrislusf/gleam - Gleam is another go based micro-footprint framework used for MR style processing. There is a parallel thread in Gleam to implement SQL layer on top of it. It would be better if octosql could have the physical planner running on Gleam