Consider how much of SourceKit-LSP’s `TaskScheduler` we could implement in the language or via executors

[ahoppen]

SourceKit-LSP implements its own custom task scheduler (https://github.com/apple/sourcekit-lsp/pull/1208) for background indexing for three reasons:

• It needs to limit the number of concurrently executing tasks. Each task will eventually just launch a process and await its result and launching significantly more processes than available cores on the machine will over-subscribe it. We get some results more quickly by scheduling these tasks in serial (because that way each indexing process is able to use an entire CPU core, if available) than if we scheduled all of the index processes concurrently (at which point they different indexing processes would fight for CPU cores and all finish later).
• We want to be able to cancel and re-schedule known long-running tasks to free up resources to execute fast-running tasks. For example, if there is an index task to index files all files in a single target with a single process but then we realize that we only need a single file from that target indexed for user-interaction, we would like to cancel the target-wide indexing task, index the single file and then come back to indexing the entire target.

I’m not sure if it makes sense to add support for either (or both) of these things to the concurrency runtime or to allow custom executors to perform these things.

[ktoso]

Thanks, even though a bit vague I think this uncovers a bunch of ideas we might want to allow:

Should an executor be able to:

• cancel a job
• inspect a job for priority (and task local values)
• know when a job "completed entirely"

Given those I think it would be possible to have a scheduler which works the way you request it to -- it keeps track of its jobs and runs them until completion with some width limit etc. There's definitely pieces we can carve out of the this and improve power of executors.