search

ldo / dbussy

Python binding for D-Bus using asyncio
91 stars 22 forks source link

High CPU usage for `_reaper` #44

Closed vincentbernat closed 3 years ago

vincentbernat commented 3 years ago

Hey!

After profiling a program, I have noticed that the _reaper function is called very often and responsible for some quite CPU usage due to this. From my understanding, it is running at each tick as long as there is a DBus related task running, which is always if you are listening for signals.

I would suggest to just sleep a bit inside reaper.

  5002518   18.592    0.000  752.277    0.000 base_events.py:1815(_run_once)
  5002518    5.646    0.000  681.874    0.000 selectors.py:452(select)
  5002518  673.818    0.000  673.818    0.000 {method 'poll' of 'select.epoll' objects}
  5007678    3.841    0.000   45.105    0.000 events.py:78(_run)
  5007678    2.863    0.000   41.264    0.000 {method 'run' of 'Context' objects}
  5000867   11.825    0.000   34.350    0.000 dbussy.py:1454(_reaper)
  5003419    4.501    0.000   20.014    0.000 base_events.py:736(call_soon)
  5003419    5.591    0.000   14.493    0.000 base_events.py:765(_call_soon)
  5007620    5.966    0.000    8.111    0.000 events.py:31(__init__)
vincentbernat commented 3 years ago

Just switching from call_soon(...) to call_later(0.1, ...) fixes the issue. However, I am wondering if you could not just append a callback to the scheduled tasks to run or signal the reaper instead of having an active event loop?

ldo commented 3 years ago

The whole reason the TaskKeeper/_reaper mechanism exists is because asyncio does not keep strong refs to the task objects it creates. So they are liable to disappear at unexpected times before they complete. So the TaskKeeper base class keeps strong refs to created tasks, and the _reaper callback exists to watch for completed tasks, to get rid of the unneeded task objects.

Yeah, I should probably throttle the frequency at which the _reaper task runs. Maybe even an interval of 0.25 seconds or a second should be sufficient. Or make it caller-configurable.

Either that, or come up with a more elegant mechanism.