Closed frankmcsherry closed 9 months ago
The most recent commit reorganizes the two spine implementations, and re-exports ord.rs
's spines (the old ones) as ValSpine
and KeySpine
from their shared module root. All direct uses of e.g. OrdValSpine
are replaced with ValSpine
. The notable exception is columnation.rs
which is meant to exercise columnation and non-columnation spines, so I left it pointing at the specific ones.
Some light performance investigation suggests that this is not universally worse than the ord.rs
spine. Running the bfs
example with n=10^6, m=2x10^6, batches of size 1000, 1000 rounds completes in
ord 42.236413s
neu 39.531365375s
Running with n=10^8 and m=2x10^8, it takes both of them roughly the same time to load the data
ord 38.825862291s
neu 37.227272667s
Anecdotally through Activity Monitor, it appeared that ord
went up to about 7GB where neu
stayed around 3GB
. That was very unscientific on my part, though.
This PR provides a re-implementation of
OrdValBatch
with a few properties:Vec<(Time, Diff)>
,Vec<(Time, Diff)>
representations (RLE).There might be other things to like about it. In the fullness of time it would mean we could remove the
trace/layers
module, because the direct implementations aren't much more complicated, and in some ways are much simpler because of their directness.This PR passes tests, but it probably wants a fair bit of exercise to see if it exactly tracks the existing implementations.
cc: @antiguru