search

ferranbt / fastssz

Fast Ethereum2.0 SSZ encoder/decoder
MIT License
73 stars 44 forks source link

MerkleizeWithMixin's performance is terrible when a large `ssz-max` is specified #130

Closed patrickmao93 closed 1 year ago

patrickmao93 commented 1 year ago

Consider this struct used in ethereum beacon chain client

type ExecutionPayload struct {
        // ...
    Transactions  [][]byte  `ssz-max:"1048576,1073741824" ssz-size:"?,?" json:"transactions"`
}

The max-size of 1073741824 (2^30) makes the generated GetTree() pretty much unusable as the length Transactions can also be in the hundreds. One call could use hundreds of GB of memory.

In real life, each transaction []byte is only about several KB. Is this reasonably easy to optimize if instead of constructing the entire tree by filling the rest of the leaves with 0s, we construct a tree in which empty subtrees are represented by zero hashes at those depths?

patrickmao93 commented 1 year ago

nvm, found out this issue has already been addressed in a later versions. it's just the dependency i'm using (github.com/umbracle/go-eth-consensus) hasn't upgraded to 0.1.3 yet. I'm moving the issue there instead.