High memory usage on log burst

[kamibo]

Bug report

Required Info:

• Operating System:
  • Ubuntu 22.04
• Installation type:
  • Binaries (apt)
• Version or commit hash:
  • Humble
• DDS implementation:
  • FastDDS

Steps to reproduce issue

Launch one or many nodes that logs a lot (using functions such as RCLCPP_INFO and a 1ms timer for instance).

Expected behavior

ros2 launch should be stable regarding regarding memory usage.

Actual behavior

ros2 launch keeps on consuming memory.

Additional information

launch_service queue (asyncio.Queue()) has no limit. And only one event is handled per loop cycle (this_loop.create_task(self._process_one_event())). So the queue keeps growing until no memory left.

Note that trying to handle many events in _process_one_event seems to solve the issue.

[ZhenshengLee]

@kamibo Thanks for your report!

I also noticed about the heavy memory usage of launch.py, in Foxy version.

Could you tell me which command you use to monitor the memory usage of ros2 nodes or processes? Any recommendation tool besides top?

Thanks.

[kamibo]

@ZhenshengLee Actually I've used htop.

[HyeKaguya233]

Could you share the method you used to solve this?I also had this problem.

[kamibo]

Could you share the method you used to solve this?I also had this problem.

Actually I'm not sure about the proper way to fix this. But you can try to add a while loop around message handling in _process_one_event.

launch_service.py

async def _process_one_event(self) -> None:                                                                                                                                                            
    while not self.__context._event_queue.empty():
        next_event = await self.__context._event_queue.get()
        await self.__process_event(next_event)   

[HyeKaguya233]

i will try,thx!!