Upcoming Priorities

Raft

Will be worked on immediately after the catalog is finished.

This will slot into executing every query. It's unlikely we'll use the the current raft crate as it is, as that was done under different assumptions regarding storage abstractions.

This next pass will have raft/sharding integrated into some of the execution nodes. Specifically when reading from or writing to a table, the planning will reference some partition map. The node responsible for reading from a partition will have raft integration such that when the plan is executed, each partition being read will undergo consensus. A similar thing will happen for writes.

Partitioning

For the first pass, each table will consist of a single shard. The second pass will partition by hash. This second pass will also include some work with partition splitting and how we update the partition map that is referenced during the planning phase.

WAL/Recovery

WAL will depend on what/how we implement raft logging. The first pass at raft will likely not have sufficient logging capabilities to act as a proper WAL. Discovery will happen during the raft implementation.

Transactions (MVCC)

Depends on how we do logging.

GlareDB / glaredb