search

hydro-project / hydroflow

Hydro's low-level dataflow runtime
https://hydro.run/docs/hydroflow/
Apache License 2.0
473 stars 33 forks source link

Add arbitrary function support for partitioning in datalog #253

Open davidchuyaya opened 1 year ago

davidchuyaya commented 1 year ago

My partitioning transformation introduces the distribution function slotPartition below.

p2a(i, no, slot, i, num, p, t') :- FilledHoles(no, slot, l, t), id(i), ballot(num, l, t), acceptors(a), slotPartition(a, slot, p), choose(_, t')

Generically, it can take any set of variables in the original output fact i, no, slot, i, num, a, t' and determine a new destination p. In Paxos, since we partition on slot, it only needs the slot number slot and acceptor a.

The function itself will be a mapping from each location a to its set of partitions p, using slot mod n to find the right partition, where n is the number of partitions.

davidchuyaya commented 1 year ago

Example pulled from here, more available in the same file.