Open gterzian opened 5 years ago
annevk
commented
5 years ago I think @domenic and I are supportive of being more disciplined and organized around this overall, but it's a rather large effort to pull off and restructure the fragile status quo.
gterzian
commented
5 years ago Thank you, and I also don't have very concrete proposals to make at this point, other than "let's try to be more aware of the issue, if we're not already".
let me get away with just one more example at this point:
I was recently asked in a review whether the "networking task-source" isn't the right one to use when queuing the navigation task as part of "following-hyperlinks". The spec uses the "DOM manipulation task-source", and it was easy enough for me to point that out.
However, if you think about it, there is little reasoning in the spec itself why it's the dom manipulation task-source, and not the networking one. For starting a navigation, one could indeed argue that the networking one could be used.
From my perspective, as explained above, I am under the impression that the networking task-source is almost always used to enqueue some result back on an event-loop, from parallel fetch steps. So from that perspective, the networking task-source is obviously not the one to use to queue a task from within the event-loop as part of "following-hyperlinks".
But then again, maybe my interpretation is wrong, or not widely shared, and tomorrow someone might well make a change to the spec, or another one, using the "networking task-source" in a way that doesn't fit into this reasoning(and apparent following of this reasoning in other parts of the spec).
What can be done? I'm not sure, but I think it would be a good thing to start discussing this explicitly, because currently the usage of task-sources and queues appears somewhat de-facto and "in between the lines"...
On another note, I think task-sources and queues are great concepts, let's have more of those! And it's amazing how logic does "fall into place" thanks to them. Just when you think there is a bug in the spec, only to realize that it's the implementation that is using the wrong task-source...
gterzian
commented
5 years ago I think it's also worth noting the current parallel-queue, used only in the context of the shared-worker-manager.
So I'd say that the task-queue of an event-loop is with regards to certain task-sources(example: networking task-source) effectively a parallel queue, from the perspective of parallel steps such as fetch, whereas the parallel-queue of the shared-worker-manager is pretty much just a task-queue for that manager, and it's only parallel from the perceptive of an event-loop.
This is where a potential definition of something like the DOM manipulation task source could differ as a "non-parallel" queue owned by a single event-loop and used to enqueue algorithms from the event-loop to it's own queue, and this would have to be enforced in terms of discipline of not using the DOM manipulation task-source to enqueue tasks from parallel steps(not saying it's necessarily a good idea, just an example of a stricter definition, and I do think the DOM manipulation task-source largely fits the bill).
Examples elsewhere are Python async queues:
asyncio queues are designed to be similar to classes of the queue module. Although asyncio queues are not thread-safe, they are designed to be used specifically in async/await code. (https://docs.python.org/3.8/library/asyncio-queue.html#queues)
Versus Python parallel queues:
a process shared queue implemented using a pipe and a few locks/semaphores. When a process first puts an item on the queue a feeder thread is started which transfers objects from a buffer into the pipe. (https://docs.python.org/3.8/library/multiprocessing.html#multiprocessing.Queue)
Obviously not saying we should define precise implementations of queues, only that we could define more about the context in which queues/sources are used, mostly parallel vs on the event-loop(and other special cases like history traversal that are a combination of event-loop(s) and parallel steps, but using in fact different queues for each although that is perhaps currently somewhat overlooked).
Question: has anyone put some thoughts into whether the task-queues and task-source concepts should be tied more closely to whether their related tasks originate from steps running "in-parallel" vs those running on the event-loop, or other exceptions?
For example, it appears to me that tasks enqueued by the dom-manipulation-task-source (almost?) always are enqueued from steps "on the event-loop", and not from "parallel" steps.
(the exception might be as part of spinning the event-loop in the-end)
On the other hand, tasks enqueued by the networking-task-source are (almost?) always enqueued from steps running "in parallel" in fetch.
(the exception might again be as part of "the-end", when processing pending-application-cache-download-process-tasks)
In other words, it seems some task-sources, like the "networking-task-source" are clearly used to "queue a task to affect the world of observable JS objects" back on the event-loop, whereas others, like the "DOM manipulation" task source, seem to be mostly about properly interleaving operations happening on the event-loop itself.
So essentially, while the spec mentions task-sources and task-queues as generic concepts, it seems they are not created equal. Some sources are mostly used to enqueue task on a queue that seems mostly be use inside a given event-loop, while others seems to be used to communicate with an event-loop from steps running in parallel of it, and others, like is discussed below, are entirely different animals.
One exception seems to be the session-history-traversal-queue, and the session-history-event-loop.
Tasks are enqueued on a session-history-traversal-queue without mentioning a task-source, so essentially tasks on it have an empty task-source.
We do have an history-traversal-task-source, but instead of being used to enqueue tasks on the "session-history-traversal-queue", it's used to queue tasks from steps running on the "session-history-event-loop" back on the "event-loop" of an agent as part of traverse-the-history-by-a-delta.
So the actual set-up seems to be something like:
Can we really still speak of generic "Task-queues", and "Task-sources" given all these de-facto differences and intricacies?
Where am I getting with all this?
I'm wondering if the task-queues and task-sources concepts could use some explicitness in terms of how their use ties into parallel vs on the event-loop vs other "cross-event-loops" exceptions like the "session-history-event-loop"(and maybe other concepts like shared-worker-manager).
Examples could be spec'ed concepts like:
A: The DOM manipulation task-source should only be used to enqueue tasks from steps running on an event-loop, if said tasks are supposed to execute on that same event-loop. B: The networking task-source should only be used to enqueue tasks from parallel fetch steps, to the event-loop where the fetch request originated from, and only as a mean to communicate updates of the related fetch request/response back to the world of observable JS objects on that event-loop.
It would seem like a good way to ensure that if someone wants to do other things with such task-sources, they would have to probably define dedicated task-sources, task-queues, and perhaps even new types of event-loops. More granularity, and details, is probably a good thing.
Finally, it would allow implementations to be more targeted, for example the DOM manipulation task-source could be tied to a thread-local only task-queue tied to a particular "event-loop", and such implementation would be robust to future spec changes. Whereas now, if you were to implement the queue used for the DOM manipulation task-source as a thread-local concept only(only related to a single event-loop), you'd run the risk that a task could be spec'ed to be enqueued from parallel steps(like is actually done in the examples at https://html.spec.whatwg.org/multipage/#spin-the-event-loop).