Please note that we do not intend to merge this PR, in only exists as documentation of the work done, and as a base for further developments if warranted.
Resolves #974.
This change is mostly done in two separate parts:
Boost interprocess memory occupancy tracking
The Boost interprocess library (which is used to manage memory within the shared memory segment allocated by chainbase) can use multiple memory allocation managers.
Chainbase, and therefore nodeos, use the rbtree_best_fit algorithm which keeps track of free areas using a rbtree (the nodes of the rbtree are intrusively stored in the free areas themselves, so there is no additional memory cost for this tree, besides the fact that it requires a minimum allocation size).
This PR adds a new member in the memory segment header managed by the interprocess library, of the type:
using occupancy_array_t = std::vector<uint8_t>; // one byte describes the occupancy percentage of each page of the segment
and updates this occupancy vector:
when a new function bool initialize_occupancy(); is called
when memory is allocated or freed
As we cannot add these changes in the original boost repo, this PR links to a private repo which includes the changes in the interprocess library. These changes are also included in the following file:
Use the occupancy vector maintained by Boost interprocess to update a texture and visualize it on-screen
The visualization is implemented using OpenGL. This required adding some dependencies to chainbase:
some glad header files describing the OpenGL interface used. These were generated from https://gen.glad.sh/, specifying gl 4.6 and glx 1.4, compat profile, header only, GL_ARB_direct_state_access extension.
the glfw library, with this trivial change:
glfw.txt
The visualization is implemented in the new files mem_visualizer[.hpp, .cpp], and takes place on a separate thread.
Notes
when using the rbtree_best_fit memory allocator (as chainbase does), the minimum size of a block is 48 bytes. When allocating 8 bytes, the overhead is 40 bytes. When allocating 32 bytes or more, the overhead is 16 bytes.
the minimum 16 bytes overhead is for two sizes (m_prev_size and m_size) with two 1 bit flags encoded in the lower two bits of m_size (m_prev_allocated and m_allocated).
Please note that we do not intend to merge this PR, in only exists as documentation of the work done, and as a base for further developments if warranted.
Resolves #974.
This change is mostly done in two separate parts:
Boost interprocess memory occupancy tracking
The Boost interprocess library (which is used to manage memory within the shared memory segment allocated by chainbase) can use multiple memory allocation managers.
Chainbase, and therefore nodeos, use the
rbtree_best_fit
algorithm which keeps track of free areas using a rbtree (the nodes of the rbtree are intrusively stored in the free areas themselves, so there is no additional memory cost for this tree, besides the fact that it requires a minimum allocation size).This PR adds a new member in the memory segment header managed by the
interprocess
library, of the type:and updates this occupancy vector:
bool initialize_occupancy();
is calledAs we cannot add these changes in the original boost repo, this PR links to a private repo which includes the changes in the
interprocess
library. These changes are also included in the following file:ip.txt
Use the occupancy vector maintained by
Boost interprocess
to update a texture and visualize it on-screenThe visualization is implemented using
OpenGL
. This required adding some dependencies to chainbase:glad
header files describing the OpenGL interface used. These were generated from https://gen.glad.sh/, specifyinggl 4.6
andglx 1.4
,compat profile
,header only
,GL_ARB_direct_state_access
extension.The visualization is implemented in the new files
mem_visualizer[.hpp, .cpp]
, and takes place on a separate thread.Notes
rbtree_best_fit
memory allocator (as chainbase does), the minimum size of a block is 48 bytes. When allocating 8 bytes, the overhead is 40 bytes. When allocating 32 bytes or more, the overhead is 16 bytes.m_prev_size
andm_size
) with two 1 bit flags encoded in the lower two bits ofm_size
(m_prev_allocated
andm_allocated
).