dreamcatcher-tech / dreamcatcher-stack

https://dreamcatcher.land
1 stars 0 forks source link

Make a HAMT array with optimized keying #136

Open inverted-capital opened 1 year ago

inverted-capital commented 1 year ago

The HAMT is ideal for randomized keys, but if we knew ahead of time that we were using an array, then the key hashing can be improved. The main advantage is speed since less buckets would be updated for common array operations like push(), and less bucket reading would be required to iterate over the array.

Buckets are assigned numerically, and the arrays are sparse by default.

As an extension other array operations can be provided with speed. Slice can be very efficient, trimming the length to any value is near instant.

Further, array operations can be multithreaded and spread across machine if required. find() can be done in parallel across many machines, filter could be split up too, forEach and map can be sharded, reduce can be in parallel, and possibly sort can be tackled in isolated segments

inverted-capital commented 1 year ago

This could be encapsulated in chainland by making a special chain that represents an array. It may segregate its children in buckets, and use these buckets or groups of buckets as a way to apply parallel computing resource against the children.