ralexstokes / ssz-rs

Implementation of ethereum's `ssz`
Apache License 2.0
102 stars 40 forks source link

Cache hash tree root computation #17

Open ralexstokes opened 2 years ago

ralexstokes commented 2 years ago

some design questions:

which types?

List and Vector are good targets -- the other custom types provided here are small enough to not need caching given the complexity it would entail (lots of wrapping etc)

An interesting one is containers with the SimpleSerialize proc macro...

One option is an attribute macro to change the struct definition in-place

Another option is to modify the derive macro so that it has a helper attribute and then it is on the caller to include the cache as needed (using the attribute to signal to the derive macro it should be used when computing the hash tree root).

I personally lean towards this last option at the moment as it is explicit and gives the caller the most flexibility on how to add caching to their custom types.

mutability model?

interior or exterior mutability? the cleanest thing would be to encapsulate all of the mutable caching behind an immutable type

relevant data to consider:

ralexstokes commented 1 year ago

partial implementation deleted in the one commit of #74

could be a good starting place to pick this back up when perf becomes an issue

ralexstokes commented 5 months ago

see #6 for an initial direction here

ralexstokes commented 5 months ago

some current thoughts:

1) consider putting the cache behind an immutable wrapper, in which case HashTreeRoot::hash_tree_root could take an immutable reference as well -- this has implications for all the consumer crates of this one so it would imply quite a substantial code refactor down the line...

2) a "lightweight" version of caching here just caches the root of the tree and marks any leaf elements that are "dirtied". a more substantial version would cache the entire Merkle tree alongside the data structure to make the hash tree computation even more efficient in terms of compute. an even more sophisticated option would support "journaling" writes so it becomes easy to undo mutations.