Open AGWA opened 5 years ago
A data structure like this might be useful:
type DiscontiguousCollapsedTree []CollapsedTreeSegment
type CollapsedTreeSegment struct {
Index uint64
Tree CollapsedTree
}
func (t *DiscontiguousCollapsedTree) Add(index uint64, tree CollapsedTree) {
maxSize := uint64(1) << bits.TrailingZeros64(index)
if tree.size > maxSize {
// error: tree too large to be placed at this index
}
// find segment prior to index in *t. append tree to this segment if tree is contiguous (and also see if we can merge in the following segment), otherwise insert a new segment in *t at this location
}
It's possible to obtain significantly higher throughput by downloading entries in parallel. There are several challenges with this approach, however: