search

protolambda / remerkleable

Typed mutable SSZ views over cached and immutable binary merkle trees
MIT License
23 stars 9 forks source link

Possible Memory Leak from Subview Retention #10

Open wemeetagain opened 3 years ago

wemeetagain commented 3 years ago

A sub-View contains a reference to the parent View, used to propagate updates to the parent. If the subview is kept around, this reference prevents the parent from being garbage-collected. You may look at making the reference to the parent a "weak reference", if python allows that sort of thing.

Haven't looked too deep to see if this is an actual issue here, I just know our implementation is based on remerkleable, so seems pertinent.

See https://github.com/ChainSafe/persistent-merkle-tree/pull/26 for our proposed fix.

protolambda commented 3 years ago

Thanks for the report. Generally subviews are meant to mutate the subtree in the super view. The GC is indeed problematic views are cached or retained somewhere. The way around it is to not retain sub-views unnecessarily: views are more like temporary typed interfaces, and the bare tree of nodes is wrapped with a view whenever necessary, and the view is then thrown away again when you are done with it. I'd only really retain the view around the beacon state as a whole, and throw away any smaller views (or alternatively "copy" them to un-hook from parent).