Memory consumption increases over time

[siferati]

Description

When voxels are expired, they are set to inactive in the underlying grid by calling openvdb::DoubleGrid::Accessor::setValueOff

https://github.com/SteveMacenski/spatio_temporal_voxel_layer/blob/6077efe368780030eabd47835f103f256836b140/src/spatio_temporal_voxel_grid.cpp#L483

The problem with this is that the voxel still remains in the grid. This means every voxel, even if inactive, will stay in memory for as long as STVL is running.

The only time when memory is freed is when the clear costmap service is requested and openvdb::DoubleGrid::clear is called.

https://github.com/SteveMacenski/spatio_temporal_voxel_layer/blob/6077efe368780030eabd47835f103f256836b140/src/spatio_temporal_voxel_grid.cpp#L426

Note that even the clear costmap recovery behavior doesn't free the memory, since that recovery behavior calls costmap_2d::CostmapLayer::clearArea, which also simply sets the voxels to inactive.

https://github.com/SteveMacenski/spatio_temporal_voxel_layer/blob/6077efe368780030eabd47835f103f256836b140/src/spatio_temporal_voxel_grid.cpp#L457

This means that under regular operation, where the clear costmap service isn't usually called, stvl memory consumption will keep increasing over time as the robot explores new areas of the map and sees new obstacles.

If you monitor the memory consumption with htop it's easy to see it increasing as the robot moves.

Demo

Thankfully, there's a function to print the memory being used by the grid openvdb::v6_2::DoubleGrid::memUsage, so if you simply log that information the problem becomes obvious.

For example, add the following lines to the top of SpatioTemporalVoxelGrid::ClearFrustums function:

setlocale(LC_NUMERIC, "");
ROS_ERROR("memory: %'lu", _grid->memUsage());

You can see how even though the obstacles (stvl/voxel_grid topic) are gone in rviz, the memory still keeps increasing over time.

https://github.com/SteveMacenski/spatio_temporal_voxel_layer/assets/15384781/00b11a9d-d68e-43ca-ba9c-505d289494b7

Solution

I looked through openvdb api and it doesn't seem like there is a way of removing a specific voxel after inserting it? But we can achieve the desired results by calling openvdb::Grid::prune.

• I'm not sure if this is the best solution, but let's discuss in #271.

[SteveMacenski]

We'll discuss in the PR, but that seems reasonable. Pruning is new in 2022 so that wasn't something we had access to back in 2018 when this was made :-) I think also that the clear operation did free the memory, I did extensive testing when I built this to make sure it didn't grow needlessly -- my guess is the API changed and we didn't change with it in this project

[SteveMacenski]

Merged in ROS 1 and will merge shortly in ROS 2 after I manually test