dat-ecosystem-archive / book

Documentation on how to implement the Dat Protocol [ DEPRECATED - see https://github.com/hypercore-protocol/new-website/tree/master/guides for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
https://datprotocol.github.io/book/
Apache License 2.0
68 stars 9 forks source link

Add optimization for full_roots function #32

Open smoyer64 opened 3 years ago

smoyer64 commented 3 years ago

**🙋 feature request

I started my career as an embedded systems engineer which meant that I had a long opportunity to become an expert at bit-wise operations. I've created an algorithm that dramatically improves the performance of finding (or in this case calculating) the full roots for a flat-tree - the reference implementation (in Javascript) iterates over all nodes in the tree with an O(n) performance while the new algorithm is based on the number of bits needed to store the record count and has O(n * log2(1)) performance.

If you'd like a paragraph describing this optimization, feel free to assign this issue to me.

Possible Solution

Additional optimization is possible as the roots for a tree only change when records are appended. It's also possible to cache the root list for a tree of a given size after calculating it and then use that value for any tree of the same size.

Code Sample

Here's my Go code for calculating the list of full roots for a flat-tree. Note that the size of the underlying array is retrieved as len(t.Nodes) but it would be possible to create the same function with a signature of func Roots(size int) []int to match the signature of the Rust example given in the book.

/*
Roots calculates the list of "full roots" for the FlatTree.  
*/
func (t FlatTree) Roots() []int {
    recs := (len(t.Nodes) + 1) >> 1
    roots := []int{}

    // If there are an odd number of leaf nodes (records) then the last
    // record is also a root and can be calculated as the value of the
    // other set bits times two.
    if recs&1 != 0 {
        recs ^= 1
        roots = append(roots, recs<<1)
    }

    // For the rest of the set bits, the root can be calculated as the
    // as the sum of the current bit minus one plus the value of the
    // other set bits times two.  If the current bit is zero, the loop
    // short circuits.
    for bit := 2; recs > 0; bit <<= 1 {
        if recs&bit == 0 {
            continue
        }

        recs ^= bit
        roots = append([]int{(recs << 1) + (bit - 1)}, roots...)
    }

    return roots
}