lbryio / lbrycrd

The blockchain that provides the digital content namespace for the LBRY protocol
https://lbry.com
MIT License
2.57k stars 178 forks source link

Properly manage the allocation of CClaimTrieNode #158

Closed kaykurokawa closed 5 years ago

kaykurokawa commented 6 years ago

CClaimTrieNode (nodes on a claimtrie) is currently allocated/de-allocated through the new/delete operators which has caused some problems in the past: https://github.com/lbryio/lbrycrd/pull/94

Using shared pointers would be a good improvement over this, so this could be the first step on this issue. Furthermore, we need to restructure the code so that we have a fixed size memory pool that limits the number of CClaimTrieNode objects that can be loaded into memory, while the rest is kept on disk (at this point the code will create CClaimTrieNode objects until it runs out of memory)

This PR was started to solve this problem https://github.com/lbryio/lbrycrd/pull/100 using the PIMPL idiom but was left unfinished. I think it has the right idea, but it ran into a bug that we could not resolve. Further work on this should at least involve looking over this PR to see if it is a good approach, and suggest alternative approach if not.

Acceptance Criteria

1. 2. 3.

Definition of Done

bvbfan commented 6 years ago

Ill investigate, if you have some valgrind output will be helpful. So i can take this issue, if you have priority work to do.

bvbfan commented 6 years ago

I've done it in this way:

  1. All CClaimTrieNode allocations are on stack
  2. None takes ownership of CClaimTrieNode* but makes exclusive copy on the heap
  3. Do not add same pointers on different containers, unless you use boost::shared_ptr
  4. Removing from container should deletes resource, unless you use boost::shared_ptr
bvbfan commented 6 years ago

I have a implementation in my local branch that resolve issue, i'll write a little unit test to check number of destroyed nodes.

alyssaoc commented 6 years ago

@bvbfan Could you let me know the status of this, please?

bvbfan commented 6 years ago

It's on hold, it waits normalization and upstream merge.