bharel / asynciolimiter

A simple yet efficient Python AsyncIO rate limiter.
MIT License
17 stars 2 forks source link

"Event loop is too fast" #14

Open davidhyman opened 1 month ago

davidhyman commented 1 month ago

Using the asynciolimiter.Limiter I'm hitting this assertion, unsure what's to be done about it;

site-packages\asynciolimiter\__init__.py", line 356, in _wakeup
    assert -leftover_time < self._time_between_calls, (
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Event loop is too fast. Woke up 1.9999999979821346 ticks early.

On Python 3.12, rate limit of ~166.666 with 156 tasks

bharel commented 1 month ago

Oh that's intriguing. It somehow means that the event loop is scheduling for sooner than it is told.

I need to think of the implications. Maybe it's due to clock or float precision.

bharel commented 1 month ago

Uhhhh... Technically it's a bug in Python or the operating system I believe...

That assertion is not too harmful but does indicate a highly inaccurate system clock or a different event loop issue that will increase CPU utilization. It won't run on py -o which you would usually use in production.

If you wonder, it all boils down to loop.call_at actually waking up earlier than told 🤦

I might remove it and change to a warnings.warning (ResourceWarning) if it's more than 2 clock ticks or 1 limiter tick early.

@davidhyman Would you mind removing that assertion locally and switching to logging.warning?

Tell me how often does that warning actually pops up, the operating system you run on and if you're using any alternative event loop implementation or event loop timer (uvloop etc.)

bharel commented 1 month ago

Are you using Windows x64 by any chance? Seems like there's an issue at python/cpython#110695 with monotonic clock accuracy causing call_at to be scheduled earlier than two clock ticks which may exacerbate the problem.