Closed fknorr closed 1 month ago
Check-perf-impact results: (7b849de16ff11660b98988ab0b032db7)
:question: No new benchmark data submitted. :question:
Please re-run the microbenchmarks and include the results if your commit could potentially affect performance.
Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | ||
---|---|---|---|---|---|
include/async_event.h | 7 | 11 | 63.64% | ||
include/thread_queue.h | 52 | 57 | 91.23% | ||
<!-- | Total: | 84 | 93 | 90.32% | --> |
Files with Coverage Reduction | New Missed Lines | % | ||
---|---|---|---|---|
include/utils.h | 2 | 89.74% | ||
<!-- | Total: | 2 | --> |
Totals | |
---|---|
Change from base Build 9661291153: | -0.07% |
Covered Lines: | 7132 |
Relevant Lines: | 7357 |
Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | ||
---|---|---|---|---|---|
include/async_event.h | 7 | 10 | 70.0% | ||
include/thread_queue.h | 52 | 57 | 91.23% | ||
<!-- | Total: | 84 | 92 | 91.3% | --> |
Files with Coverage Reduction | New Missed Lines | % | ||
---|---|---|---|---|
include/utils.h | 2 | 89.74% | ||
<!-- | Total: | 2 | --> |
Totals | |
---|---|
Change from base Build 9661291153: | -0.05% |
Covered Lines: | 7132 |
Relevant Lines: | 7356 |
Why does OoO dispatch alleviate the need for a thread pool?
Because OoO assignment addresses individual threads as lanes, whereas before, the thread pool decided which job would get assigned to which pool thread. The backend will simply maintain an array of thread_queues
and explicitly index into it with the lane id.
This PR upstreams and tests two threading facilities required for IDAG execution: The in-order
thread_queue
and thedouble_buffered_queue
for fast communication between threads.async_event
gains the ability to return results from async operations (in practice always pointers = results fromsycl::malloc
, which is why I opted to make the result typpevoid*
instead ofstd::any
)thread_queue
takes functors as submissions and returnsasync_events
while providing timing/profiling capabilities for host task, host copy and allocation operations. This will replace the CTPL thread pool, which would require a lot of boilerplate to keep in while providing little benefit, because with OoO dispatch, we don't have a need for pools with > 1 thread anymore.double_buffered_queue
generalizes the queueing mechanism that's currently in place between main- and scheduler thread and ensures the consumer is contended as little as possible. This is a stand-alone component now because it will also be used between scheduler and executor threads.I do realize there's a staggering number of things called queue now.