search

martinsumner / kv_index_tictactree

Provide Active-Anti-Entropy features to a virtual node in a KV store
Apache License 2.0
21 stars 7 forks source link

Control logging of state #102

Open martinsumner opened 2 years ago

martinsumner commented 2 years ago

Use format_status/2 in gen_server so as not to log large amounts of state in crash dumps

martinsumner commented 2 years ago

Related to this is the problem of the change_queue in the aae_keystore.

This queue builds up all puts seen since the rebuild of the aae_keystore was started. This can end up being very large, if there are a lot of PUTs happening concurrently to the rebuild. There is also the possibility that rebuilds could be occurring concurrently on a node.

The queue could cause a node to run out of memory, and there is no way to maintain the functional integrity of the store if the queue is limited.

Preference is therefore to switch to using disk_log for this queue. This will be slower, but means there is no risk with memory exhaustion during the rebuild.

martinsumner commented 2 years ago

https://github.com/martinsumner/kv_index_tictactree/pull/103