helium / blockchain-core

Apache License 2.0
214 stars 86 forks source link

Revert "Try not deleting the hex cache, just delete from pdict" #1467

Closed PaulVMo closed 2 years ago

PaulVMo commented 2 years ago

This reverts commit 22302c97c48ea27380086dd284de268609ecce62 in attempt to fix a memory leak. While in consensus group, the memory usage for a validator groups and does not return to pre-consensus levels once elected out. I tracked this to large ETS usage after CG (e.g., 3.8 GB on a validator which had a CG run since last reboot which is 10X that of a validator that has not been in CG since reboot). Much of that space is occupied multiple copies of __blockchain_hex_unclipped_tbl and __blockchain_hex_clipped_tbl (one each per epoch in CG).

id                                  name                             type  size    mem     owner
 ----------------------------------------------------------------------------
#Ref<0.4233474657.597819418.185503> '__blockchain_hex_unclipped_tbl' set   2395389 23961932 miner
#Ref<0.4233474657.597819418.185504> '__blockchain_hex_clipped_tbl'   set   2395389 23961932 miner
Vagabond commented 2 years ago

This is probably correct, but I don't remember why I made this change.

xandkar commented 2 years ago

This is probably correct, but I don't remember why I made this change.

IIRC you were experimenting with taking greater advantage of the cache in node or something else that was also using core.