imnotjames
commented
11 months ago Code snippet of a first pass of implementation:
```python
from heapq import heappush, heappop
class TaskQueue:
def __init__(self):
self._heap = []
self._entries = {}
self._counter = 0
def peek(self):
while self._heap:
task = self._heap[0][-1]
if task is not None:
return task
heappop(self._heap)
return None
def push(self, task: Task, key=None):
if key is None:
key = core.ticks()
task.ph_key = key
task.data = None
self._counter += 1
entry = [
key,
self._counter,
task
]
self._entries[task] = entry
heappush(self._heap, entry)
def pop(self):
while self._heap:
key, _, task = heappop(self._heap)
if task is not None:
del self._entries[task]
return task
return None
def remove(self, task: Task):
# Find the task and tombstone it
if not task in self._entries:
return
entry = self._entries.pop(task)
entry[-1] = None
# Compatibility aliases, remove after they are no longer used
push_head = push
push_sorted = push
pop_head = pop
```
dhalbert
I have a hypothesis that the big reason why
TaskandTaskQueueare implemented in C are because the task queue's pairing heap implementation in C is far more performant than implementing it in native python.Because the entire TaskQueue is in C, with the way it's written it needs to reference the tasks themselves - so then the Task class has to be written in C as well. This leads to troubles when - for example - we want to reference other errors, make changes to the Task class, etc. (Okay, so maybe it's just me that has been feeling this pain!)
Something I was poking at is rewriting the
TaskQueueto use theheapqimplementation. The big question I have and why I opened this issue is to ask "Is this a path forward?"Another alternative would be to abstract the
Taskpiece so it no longer stores its ownph_key/ etc and include the task within the pairing heap wrapped. There's two places whereTask'sph_keyis checked outside of the task queue - once by the task itself during cancellation (this could be done by somehow getting when a task is scheduled from the currently running loop?) - and once by the running loop which could just be asking the task queue when the next Task is going to be at during the peek.