rmw_cyclonedds version - 1.1.1 cb129b5859e4c1793757d71f521540ae67a4fb20
DDS implementation:
rmw_cyclonedds_cpp
Client library (if applicable):
rclcpp
Steps to reproduce issue
Set up a pub/sub example with all the required SHM constraints
Make the subscriber callback significantly time-consuming
After some iterations, the shared memory chunks pile up and eventually leading to"out of chunks" error.
Actual behavior
When the subscriber callback is expensive and takes significant time to complete, the shared memory chunks are lost
Expected behavior
The shared memory chunks shouldn't be lost in scenarios where the subscriber callbacks take significant time to execute and the chunks should be freed properly.
Additional information
The probable reasoning for why this happens is as follows, when shared memory is enabled, the flow for executing the subscription is first the loaned message is taken, then the callback is executed with the taken loaned message, then once the callback execution is completed the loaned message is returned back to the middleware. However, if the callback is expensive and takes significant time, then in the mean time there are samples also received in the reader history cache, and based on the history depth the samples in the reader history cache will be lost without being processed by the executor. Ideally, when the sample is removed from the reader history cache, the corresponding loan for this sample should also be returned to the middleware, which is not handled correctly in serdata_rmw_free.
The essence is in serdata_rmw_free the loan is not returned to the middleware if there is an outstanding loan.
Bug report
Required Info:
rmw_cycloneddsversion - 1.1.1 cb129b5859e4c1793757d71f521540ae67a4fb20rmw_cyclonedds_cpprclcppSteps to reproduce issue
Actual behavior
When the subscriber callback is expensive and takes significant time to complete, the shared memory chunks are lost
Expected behavior
The shared memory chunks shouldn't be lost in scenarios where the subscriber callbacks take significant time to execute and the chunks should be freed properly.
Additional information
The probable reasoning for why this happens is as follows, when shared memory is enabled, the flow for executing the subscription is first the loaned message is taken, then the callback is executed with the taken loaned message, then once the callback execution is completed the loaned message is returned back to the middleware. However, if the callback is expensive and takes significant time, then in the mean time there are samples also received in the reader history cache, and based on the history depth the samples in the reader history cache will be lost without being processed by the executor. Ideally, when the sample is removed from the reader history cache, the corresponding loan for this sample should also be returned to the middleware, which is not handled correctly in
serdata_rmw_free.The essence is in
serdata_rmw_freethe loan is not returned to the middleware if there is an outstanding loan.