Open jcarraror opened 1 year ago
Branch 27-periodic-save
(commit 99a6f35) contains a prototype for a possible implementation of Storage::periodicSaveToDB()
. Heavy emphasis on prototype.
The thread is already set to start at the end of Storage's constructor and stop at the start of its destructor. It starts just fine and the cooldown is iterated upon as well (every 15 seconds), but for some reason I don't see the message about stopping the thread when I kill tmux. Unless I'm missing something here, that means the destructor is not being called at all, which also means the blockchain is never really being saved at all, independent of the periodic thread, and this actually concerns me since it practically means we have no idea if the destructor is actually working...? Either way, you can check this yourselves in debug.txt
, filtering by "periodic".
Basic logic for the saving routine is as follows (subject to change, could use some input on the whole thing, including testing which was not done properly due to lack of feedback on how to properly test it):
cachedBlocks
has at least 1000 blocks, or cachedTxs
has at least 1M (1 million) transactions - start the saving routine if that's the case, skip it and wait 15 seconds for the next loop otherwiseshared_ptr::use_count()
(check the code comments for more info) - if not, wait until it is (checking every 1ms on a loop)How could this be properly tested? The routine only kicks in at 1000 blocks or 1M txs on cache (which is also weird because we should be checking the chain and I still don't get that, but I strictly followed the TODO instructions anyway), so we'd need to check the following steps:
debug.txt
all the way
Motivation: Currently, all blocks of the blockchain are being stored to the memory, we must add functionality to periodically dump the blocks into the database in order to not overload the system memory with blocks
Objectives: Periodically dump the most old unused blocks from memory to the database
Success Criteria: Safely dump old blocks to the database, not locking Storage for too long (more than one second) while dumping.
Dependencies: No external dependencies.
Risks and Mitigations: It is necessary to be aware of blocks being a shared_ptr’s, meaning that while it is being used by another thread, it should not be dealocated from memory, besides that, it should not lock the storage for too long where it would create a deadlock within the node, limiting it’s response for processing new blocks or requests from other nodes, besides that, priority should be given to the older blocks within the deque.
Resource Allocation: This milestone will deeply affect the Storage function