In ipld-hashmap there are some inefficiencies when it interacts with the store. These can be sped up by:
Allow a store that stores Blocks directly, rather than byte arrays by having Load and Save test if the store provided a Blocks interface, to avoid dealing in raw bytes and the rehashing and re-encoding that is required.
Make the hasher have a weakmap as a memoization cache
Save should cache with a weakmap or some other dedupe in case the bucket had been encoded before. This would be superseded by a blocks aware store.
The HAMT implementation appears outdated for speed, and can be improved by removing the putStore completely. This would mean:
.crush() could reuse the existing hashmaps without calling .load() every time it ran.
diff blocks would be found by walking new links and reporting all CIDs along the way, which should be very quick and needs no complicated code at all.
Remove the need for putStore diffing first, then make putStore act like simply a puts buffer during crush. It should have a lock when crushing begins to cause an error or wait if any other crush is occuring simultaneously.
Ultimately switching is the bigger performance enhancement, especially for under the threshold usage.
In ipld-hashmap there are some inefficiencies when it interacts with the store. These can be sped up by:
The HAMT implementation appears outdated for speed, and can be improved by removing the putStore completely. This would mean:
.crush()could reuse the existing hashmaps without calling.load()every time it ran.Remove the need for putStore diffing first, then make putStore act like simply a puts buffer during crush. It should have a lock when crushing begins to cause an error or wait if any other crush is occuring simultaneously.
Ultimately switching is the bigger performance enhancement, especially for under the threshold usage.