python / asyncio

asyncio historical repository
https://docs.python.org/3/library/asyncio.html
1.04k stars 177 forks source link

Trim off finished futures from the Event._waiters #431

Closed ZhukovAlexander closed 7 years ago

ZhukovAlexander commented 7 years ago

Current implementation of Event.wait uses dequeue.remove(e), wich has O(n) complexity. We can do better by utilizing this time to also remove all finished futures from the queue on the way up until we reach the future that is in a pending state. This will make subsequent calls to Event.wait or Event.set to perform less work on a waiters queue

sethmlarson commented 7 years ago

Pretty sure that fut is always at index 0 within _waiters by the time that it yields. remove() only removes the first encountered element so it'll stop there, so probably no speed-up using this trimming method. Instead of using remove() could just use popleft() as it's probably faster.

ZhukovAlexander commented 7 years ago

@SethMichaelLarson , I think you are right. Though, the side effect of this algorithm is that it will also trim all finished futures that come after the current one, so it still has an effect.

sethmlarson commented 7 years ago

@ZhukovAlexander When would that occur? Each future trims itself as it's finished so there's only ever one finished future in _waiters. The only time I can think where that would happen is if code was muddling with the internals.

gvanrossum commented 7 years ago

If fut always occurs at index 0 there is no O() improvement from using popleft(), so the point of the diff is moot -- let's close this PR then.