Commands that contain large binary data (as e.g. in RabbitMQ quorum queues)
can cause undesireable GC behaviour in the Ra leader (and followers) as
commands are maintained in the process in two places:
A low priority command queue that is used to hold back
lower priority commands in favour of normal priority ones.
(In quorum queues this means messages vs acks).
The Ra log holds on to entries until it gets a confirmation
from the WAL that the entries have been written and are now
available in the open mem table.
This commit includes a low priority queue that keeps the commands
in a private ETS table which is read as the queue is consumed.
The log cache works similarly flushing entries to a private ETS table
at end of each incoming erlang message.
The end result is that large binaries can be much more readily collected
and will be much less likely to be promoted to the old heap and/or trigger
preemptive major collections due to reaching the vheap size.
ETS backed log cache and low priority queue.
Commands that contain large binary data (as e.g. in RabbitMQ quorum queues) can cause undesireable GC behaviour in the Ra leader (and followers) as commands are maintained in the process in two places:
This commit includes a low priority queue that keeps the commands in a private ETS table which is read as the queue is consumed.
The log cache works similarly flushing entries to a private ETS table at end of each incoming erlang message.
The end result is that large binaries can be much more readily collected and will be much less likely to be promoted to the old heap and/or trigger preemptive major collections due to reaching the vheap size.