Add support for detecting page faults within async/await coroutines

[unicomp21]

Something similar to structured exception handling which can bubble up along the async/await exception handling chain. Possible use case:

Task taskHasPageFaulted = Task.RunInPageFaultScheduler(MyCoroutine);

Task nextPageFaultedCoroutine = await Task.WaitAny(arrayOfAboveCoroutines); // run a coroutine which is ready

The dream setup would be using UMS (User Mode Scheduler) from .Net Core. UMS nailed the page fault thing, when combined w/ structured exception handling.

The intention is being able to multiplex 100k+ coroutines efficiently.

[benaadams]

You can cooperatively give up execution in a Task with await Task.Yield(); the default scheduler should happily handle 100k+ items; however you might want to look at implementing a custom System.Threading.Tasks.TaskScheduler

e.g. you could create one with its own threads (rather than threadpool) with:

private sealed class DedicatedThreadsTaskScheduler : TaskScheduler
{
    private readonly BlockingCollection<Task> _tasks = new BlockingCollection<Task>();

    public DedicatedThreadsTaskScheduler(int numThreads)
    {
        for (int i = 0; i < numThreads; i++)
        {
            new Thread(() =>
            {
                foreach (Task t in _tasks.GetConsumingEnumerable()) TryExecuteTask(t);
            })
            { IsBackground = true }.Start();
        }
    }

    public Task Run(Func<Task> func) => StartNew(func).Unwrap();

    protected override void QueueTask(Task task) => _tasks.Add(task);

    protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued) => TryExecuteTask(task);

    protected override IEnumerable<Task> GetScheduledTasks() => _tasks;
}

Then create it as

static TaskFactory scheduler = new TaskFactory(
    CancellationToken.None,
    TaskCreationOptions.HideScheduler, TaskContinuationOptions.HideScheduler,
    new DedicatedThreadsTaskScheduler(StartThreadCount));

And queue the entry-point tasks with

await scheduler.Run(() => MyAsyncFunc()).ConfigureAwait(false);

Integrating with User-Mode Scheduling and its alerts on blocking system call or a page fault is interesting; and you could have it trigger next in queue in scheduler; though it would need its own UMS worker threads and you could have UmsSchedulerProc be a call back into the TaskScheduler.

Interested in what you discover...

/cc @stephentoub

[benaadams]

Some info on custom task schedulers:

• StaTaskScheduler
• ConcurrentExclusiveInterleave
• Additional TaskSchedulers

[omariom]

Quoting MSDN:

The system also allows a thread waiting in GetQueuedCompletionStatus to process a completion packet if another running thread associated with the same I/O completion port enters a wait state for other reasons, for example the SuspendThread function.

When a thread encounters a hard page fault it enters wait state. It (theoretically) means that at least I/O part of the thread pool already handles page faults in a way.

[unicomp21]

Thanks @omariom, I didn't know GetQueuedCompletionStatus could be leverage. Sounds like a much simpler solution. Is there anything similar for epoll on linux?

[unicomp21]

@benaadams thanks for the great feedback, much appreciated. Couldn't help but notice the webgl part of your profile. Any luck w/ the gltf thing? I've been looking at babylon.js, but my perception is the mikkt support isn't spot on. In addition, I'd love to use webassembly. From where I stand right now kotlin w/ ts2kt is looking pretty promising. Any suggestions?

[stephentoub]

Closing as asked/answered. Please re-open if there's something remaining I missed. Thanks.