Open dicej opened 4 months ago
Tagging subscribers to this area: @mangod9 See info in area-owners.md if you want to be subscribed.
cc @agocke
In Mono/Browser when single-threaded, we have thread-less thread pool implemented via emscripten Module.safeSetTimeout
which is just wrapper around JavaScript setTimeout
.
This has benefit of yielding to the browser event loop.
Yielding is necessary so that JS Promise
s could get resolved, for example fetch
.
Hope this helps.
I would like to learn more on how yielding out of the current (dotnet) component need to look like for WASI p2/p3.
(Just thinking aloud)
The list of "jobs" we have in the thread-less thread pool solves all non-external Task
s and their dependencies.
Also note that any locks are NOOP in our single-threaded build.
And the browser (host) is calling back to us when there is setTimeout
tick or when JS promise gets resolved.
Do I understand it right that in WASI instead of being called by host on "event" we are expected to have loop in some main()
and do poll()
on all external promises.
Meaning that WASI promises will become resolved by the WASI host even without us yielding ?
Would that poll loop still work in preview 3 too ? Would that poll loop still work in multi-threaded too ? Does it need to be re-entrant ? (dotnet calls another component synchronously, which calls back to our async export) I'm concerned about WASM stack.
Could wasi-clocks
play the role of setTimeout
? Would it help to anything ?
How would that look like if we are WASI reactor which exposes async API ?
Should all async (component) exported functions implement such poll
loop ?
To be able to exit that poll()
loop we need to know if there is no more work to do.
Probably it means that all Task
s are resolved/rejected/canceled.
And unresolved Tasks could depend on external promises, which is OK.
(For context: In browser we marshal JS Promise
as C# Task
, when the JS calls us .then(()=>{ ... callCSharp() })
we call TaskCompletionSource.SetResult
for the marshaled Task
)
I'm not sure what would be the answer if there are multiple-threads in WASI. Should we have multiple poll()
loops ?
Do WASI promises have thread affinity ? (in JS they do)
Does WASI support cancellation ? In preview 3 ?
Is pollable
concept going to survive preview3 ?
Is pollable
equivalent of a promise which could resove just once ?
Or it's a wait handle/stream/semaphore, which could be lifted multiple times ?
If multiple times, are we going to create Task
instance for each promised "event" of the pollable
?
To simplify the discussion slightly, I don't think there is any strong dependency on multi-threading with the .NET async model. It is possible that you will need to use a different default scheduler in your app model, but I don't know of any requirement that multi-threading is required for .NET async.
@stephentoub would probably be the best person to chime in on specific technical decisions that would be recommended for this area.
For reference, the Discord discussion about this starts here: https://discord.com/channels/732297728826277939/732297825731215521/1245818492322844673
Do I understand it right that in WASI instead of being called by host on "event" we are expected to have loop in some
main()
and dopoll()
on all external promises.
(@pavelsavara discussed this on Discord, but I'll answer here also for the record). Yes, WASIp2 requires the guest to run an event loop (maybe in main
, but more generally any export, e.g. wasi:http/incoming-handler
). WASIp3 will change that by using Component Model support for async imports and exports, at which point the event loop will move to the host.
Meaning that WASI promises will become resolved by the WASI host even without us yielding ?
Yes, the host can make progress on the promises regardless of if or when the guest yields.
Would that poll loop still work in preview 3 too ? Would that poll loop still work in multi-threaded too ? Does it need to be re-entrant ? (dotnet calls another component synchronously, which calls back to our async export) I'm concerned about WASM stack.
Quoting my Discord comments:
WASIp3 will be based on a major update to the Component Model which supports async imports and exports. The async exports may either be "stackless" such that the guest returns a task handle to the host and exports a callback for progress notifications, or "stackful", meaning the guest blocks on a poll-style intrinsic and supports concurrent calls on other fibers.
We expect languages that have stackless, async/await style coroutines such as JS, Python, Rust, and C# will want to use the callback approach, in which case the application developer will export async functions and wit-bindgen will handle the details of assigning a unique handle to each Promise/Task/Future and maintaining the lookup table @Pavel Savara described. I've already implemented this for Rust: https://github.com/dicej/component-async-demo/. This mechanism also allows application code to "spawn" tasks which may continue to run after the export function has returned a result (e.g. https://github.com/dicej/component-async-demo/blob/aba5ebf363d5830cbab20bf9a453f927c47e0605/http-echo/src/lib.rs#L48-L59). The host runs the top level event loop, dispatching events to the appropriate callback exports with the appropriate handles, which the guest will translate to Promises/Tasks/Futures and resume them.
Could
wasi-clocks
play the role ofsetTimeout
? Would it help to anything ?
Yeah, I was thinking the same thing, but I don't think we need to involve the host with deferred execution at all -- it can all be taken care of in some combination of the .NET runtime and wit-bindgen
-generated code.
How would that look like if we are WASI reactor which exposes async API ? Should all async (component) exported functions implement such
poll
loop ?
All exported functions that want to use async/await will need to run an event loop of some kind, but we could make that an implementation detail in wit-bindgen
so the app developer only needs to write an async
function and let the generated code add the event loop implicitly.
I'm not sure what would be the answer if there are multiple-threads in WASI. Should we have multiple
poll()
loops ? Do WASI promises have thread affinity ? (in JS they do)
It's going to be a while before WASI has threads (i.e. post-WASIp3), so I don't think we need to worry about that now. By then, the event loop(s) will be in the host instead of the guest, so the .NET toolchain won't need to deal with it anyway beyond generating thread-safe code.
Does WASI support cancellation ? In preview 3 ?
WASIp2 does not support it, and p3 probably won't either, although I've been doing design work on cancellation with @lukewagner and we plan to support it post-p3.
Is
pollable
concept going to survive preview3 ?
No.
Is
pollable
equivalent of a promise which could resove just once ? Or it's a wait handle/stream/semaphore, which could be lifted multiple times ?
It can resolve more than once. I'm not a big fan of that since it makes the host implementation more difficult, but that's how it is :)
If multiple times, are we going to create
Task
instance for each promised "event" of thepollable
?
Good question. I would say that just because you can use a pollable
more than once doesn't mean you have to, and if it makes things simpler we can just always only use it once. That's been my approach for async support in Rust and Python.
To simplify the discussion slightly, I don't think there is any strong dependency on multi-threading with the .NET async model. It is possible that you will need to use a different default scheduler in your app model, but I don't know of any requirement that multi-threading is required for .NET async.
Yeah, I don't think it's intentional, it just so happens that you currently can't use e.g. Task.WhenEach
on WASI since it always tries to queue actions to ThreadPool
, which immediately throws a PlatformNotSupportedException
. If WASI were to get a usable ThreadPool
implementation that just defers work without trying to spawn threads (which is starting to look like the best option), then that won't be a problem anymore.
@stephentoub would probably be the best person to chime in on specific technical decisions that would be recommended for this area.
I've been reading a lot of his blog posts lately while getting up-to-speed on how async
works in .NET, so yeah, I'd love to get his perspective :)
To simplify the discussion slightly, I don't think there is any strong dependency on multi-threading with the .NET async model. It is possible that you will need to use a different default scheduler in your app model, but I don't know of any requirement that multi-threading is required for .NET async.
@stephentoub would probably be the best person to chime in on specific technical decisions that would be recommended for this area.
That's right. The async/await implementation in .NET does need some kind of scheduler that work can be queued to, but it's fine for that scheduler to just be for a single thread pumping in an event loop.
I do think you'll want to reroute ThreadPool.QueueUserWorkItem and friends to just schedule to the single event loop. There's a ton of code out there, including but not limited to the async/await infrastructure, that uses ThreadPool directly, or stand-ins for it (like Task.Run), and all of that code should be able to "just work" if those work items are routed to that dispatcher: it just happens it's a pool of size 1.
Thanks for all the feedback. I'm going to try to summarize what I've read (or inferred) so far:
ThreadPool
implementation that does the right thing for that platform: queue items to be executed by a top-level, single-threaded event loop.wit-bindgen
or the .NET runtime.
async
function(s) to the host and trust that the details will be taken care of.async
function(s) should work fine in both cases.wasi:io/poll/pollable
handles into Tasks
, implicitly adding those handles to the set to be polled by the event loop.Please let me know if I've misunderstood or left out anything important.
With the above in mind, here's a concrete proposal to drive further discussion:
ThreadPool.Wasi.cs
that queues items for execution by the top-level event loop.WasiEventLoop
class with two methods:
public static T Run<T>(Task<T> task)
: Runs the event loop, which is responsible for:
ThreadPool
work itemswasi:io/poll/poll
as necessary, passing any pollable
handles which have been registeredpublic static Task Register(Pollable pollable)
: Registers the specified pollable
with the event loop, returning a Task
which will resolve when the pollable
resolves.wit-bindgen
for optionally generating async
export bindings, generating a call to WasiEventLoop.Run
for each export, passing the task returned by a call to the application-provided implementation.wit-bindgen
, specialize any WASI functions which return pollable
so that they return Task
instead. The generated code would call WasiEventLoop.Register
to perform the registration.static async void Main
functions which uses WasiEventLoop.Run
behind the scenes? I'm not sure if this would be useful as opposed to using wit-bindgen
to target wasi:cli
, but maybe?WasiEventLoop
as described above since the event loop will be run in the host, but we'll still need some way for wit-bindgen
to run any pending ThreadPool
work items, e.g. just prior to returning control to the host.Note that WasiEventLoop
is meant to be an implementation detail hidden from the application developer, although we could make it part of the supported public API if we wanted to give devs fine-grained control.
Thoughts?
If we keep WasiEventLoop
private rather than public API of the runtime, then the code generated by wit-bindgen needs to generate the unsafe accessors that SingleAccretion mentioned in the discord. I guess that's fine for now.
I think that we will also have to protect it from IL trimming (at least for Mono).
For registering external pollable, when would we free that handle ? After the Task is resolved ? Do we need to worry about pollable or p3 promises that never resolve, but the Task which was using them is GCed ?
I'm also missing the part in which dotnet Task is returned/exported as Pollable
. Could you please comment on that ?
Such pollable needs to keep the TaskCompletionSource
alive, probably via GCHandle.
What allocates the low level pollable handle (number/id) when we need to create one ?
Are we told that such pollable is not referenced anymore by the host or another component ?
For registering external pollable, when would we free that handle ? After the Task is resolved ?
Yes; I've been doing the equivalent in Rust and Python, and that's what I'm doing here: https://github.com/dicej/dotnet9-wasi-http-example/blob/da7541017f944247e7614a3cad2f663508118837/PollTaskScheduler.cs#L45-L52 -- i.e. for each ready pollable
, we first dispose of the handle and then call TaskCompletionSource.SetResult
.
Do we need to worry about pollable or p3 promises that never resolve, but the Task which was using them is GCed ?
I assume you can call TaskCompletionSource.SetResult
safely even if the corresponding Task
has been GCed, correct? I.e. it's just a no-op?
Same story for p3 promises -- the host will notify the guest when each one resolves, which will trigger a TaskCompletionSource.SetResult
, which will be a no-op if nobody is listening anymore.
I'm also missing the part in which dotnet Task is returned/exported as
Pollable
. Could you please comment on that ? Such pollable needs to keep theTaskCompletionSource
alive, probably via GCHandle. What allocates the low level pollable handle (number/id) when we need to create one ? Are we told that such pollable is not referenced anymore by the host or another component ?
The flow of control would look something like this:
OutputStream.Subscribe
(i.e. the method generated by wit-bindgen
for wasi:io/streams/output-stream/subscribe
), which is implemented by the host.pollable
handle representing write readiness for that output-stream
and returns it to the guest.wit-bindgen
-generated code, the returned handle is immediately passed to WasiEventLoop.Register
, which creates a TaskCompletionSource
and adds the (Pollable, TaskCompletionSource)
tuple to its pollables
list (e.g. https://github.com/dicej/dotnet9-wasi-http-example/blob/da7541017f944247e7614a3cad2f663508118837/PollTaskScheduler.cs#L12-L17), returning the corresponding Task
.Task
and does whatever it wants with it (e.g. await
it, call ContinueWith
on it, or whatever; presumably it will want to call OutputStream.Write
once the Task
resolves).WasiEventLoop.Run
calls wasi:io/poll/poll
next, it will pass all the pollable
handles in its list, and the host will block until at least one of them is ready.pollable
s, WasiEventLoop.Run
will dispose each of those pollable
s and call TaskCompletionSource.SetResult
on the corresponding source.pollable
that was not ready stays in the list so we can poll it again in the next iteration of the loop.See https://github.com/dicej/dotnet9-wasi-http-example/blob/snapshot/PollTaskScheduler.cs for details. Does that help?
Do we need to worry about pollable or p3 promises that never resolve, but the Task which was using them is GCed ?
I assume you can call
TaskCompletionSource.SetResult
safely even if the correspondingTask
has been GCed, correct? I.e. it's just a no-op?
Sorry, my bad. Pollable will keep TaskCompletionSource
alive and that will keep Task
alive.
Does that help?
What I mean to ask is that we have export public static Task<string> Foo()
which is wrapped as WASI export. That needs to marshal the Task
instance that C# created (from C# computation, not from pollable) into new pollable and return it to the host. When C# resolves the Task
, we need to signal the host that the pollable was resolved and pass the string
which was the promised payload.
We can pass pollable as result or as parameter of a function. And you can do it on imports and exports.
That's 4 combinations, but possibly only 2 ways how to marshal that.
Pollable
-> Task
Task
-> Pollable
I'm asking about the other direction.
See https://github.com/dicej/dotnet9-wasi-http-example/blob/snapshot/PollTaskScheduler.cs for details.
I'm not sure that you need your own TaskScheduler
, I think ThreadPool
would be enough. I'm not 100% confident, thought.
What I mean to ask is that we have export
public static Task<string> Foo()
which is wrapped as WASI export. That needs to marshal theTask
instance that C# created (from C# computation, not from pollable) into new pollable and return it to the host. When C# resolves theTask
, we need to signal the host that the pollable was resolved and pass thestring
which was the promised payload.
Ah, I see what you're saying. For WASIp2, there's no concept of guest-created pollable
s, so we can't actually return a Task
, pollable
, or anything like that to the host. Instead, the wit-bindgen
-generated code needs to call Foo
and pass the resulting Task
to WasiEventLoop.Run
, which will block until the Task
resolves. Then it can return the string
result to the host. From the host's point of view, it just called a synchronous function and got back a string.
For WASIp3, there will be a new Component Model ABI for async exports, and in that case the guest can and should return to the host if the Task
has not resolved. The host will later call the guest via a callback function when progress is made on any of the async imports the guest called. For now, though, I think we should focus on how to support WASIp2 since we're still in the design and prototype phase for WASIp3.
We can pass pollable as result or as parameter of a function. And you can do it on imports and exports. That's 4 combinations, but possibly only 2 ways how to marshal that.
Pollable
->Task
Task
->Pollable
There's no need for a Task
-> Pollable
translation since there's no way to represent a guest task/promise/future as a Pollable
in WASIp2. All exports are synchronous from the host's perspective.
See https://github.com/dicej/dotnet9-wasi-http-example/blob/snapshot/PollTaskScheduler.cs for details.
I'm not sure that you need your own
TaskScheduler
, I thinkThreadPool
would be enough. I'm not 100% confident, thought.
Yes, I think you're right. I just referred to that because I expect WasiEventLoop
will closely resemble what I wrote there.
The plan looks good to me.
Let's try it in NativeAOT LLVM branch. And we could catch up with it in Mono later.
@yowl @SingleAccretion do you have any comments ?
cc @silesmo @jsturtevant
My primary concern is layering, i. e. the question of who provides the event loop implementation (WasiEventLoop
above). My preference is for this to live under wit-bindgen
(not as generated code, necessarily, but as a supporting assembly/nuget, for example), at least for now. The main reason is that testing it would require the tool.
For reference, the interface between ThreadPool
and event loop on Browser is setTimeout(ThreadPool.RunQueuedWorkItems, ...)
(including the ability to specify the delay, used for the timer queue). It looks to me the same should work for WasiEventLoop
. @dicej is that true?
My primary concern is layering, i. e. the question of who provides the event loop implementation (
WasiEventLoop
above). My preference is for this to live underwit-bindgen
(not as generated code, necessarily, but as a supporting assembly/nuget, for example), at least for now. The main reason is that testing it would require the tool.
That works for me.
For reference, the interface between
ThreadPool
and event loop on Browser issetTimeout(ThreadPool.RunQueuedWorkItems, ...)
(including the ability to specify the delay, used for the timer queue). It looks to me the same should work forWasiEventLoop
. @dicej is that true?
I think that should be fine, yes, although I'm having trouble finding a RunQueuedWorkItems
function in the source tree. Did you mean BackgroundJobHandler perhaps?
If someone can explain to me the proper way for a hypothetical wit-bindgen
runtime helper library to call a private function analogous to ThreadPool.BackgroundJobHandler
, then I can take care of the rest. Unlike the browser case, I don't think we can make a host call for something like this -- there's no equivalent to setTimeout
in WASI, and it's not really the host's business anyway since the event loop is the guest's responsibility.
Actually, I think a better interface for WASI would be something like private static List<(Action, ulong)> TakeJobs()
which returns a list of jobs and their delays in milliseconds, which WasiEventLoop
can schedule using e.g. wasi:clocks/monotonic-clock/subscribe-duration
(or just run immediately if the delay is zero). Each call to TakeJobs
would clear the internal state of ThreadPool
, i.e. those jobs become the responsibility of WasiEventLoop
.
Did you mean BackgroundJobHandler perhaps?
Yep. It was just an example function name.
If someone can explain to me the proper way for a hypothetical wit-bindgen runtime helper library to call a private function analogous to ThreadPool.BackgroundJobHandler, then I can take care of the rest.
It doesn't really matter a whole lot what the private interface looks like mechanically. For example, it can be a private static method SetEventLoopQueueFunction(delegate*<delegate*, ...> pQueueFunc)
that would set the queue function on the thread pool, with wit-bindgen being expected to call that function somewhere on the startup path.
The main concern is that the interface be small. The thread pool's queue internal machinery is pretty involved. It seems unlikely that we will be able to fit through something like TakeJob
s seamlessly (and without losing fidelity, e. g. diagnostics).
Sounds good. How would the wit-bindgen
helper library call SetEventLoopQueueFunction
then? @pavelsavara mentioned unsafe accessors and IL trimming, but I don't know what that translates to concretely.
Is the WasiEventLoop
ideal place to schedule finalizers?
Is the WasiEventLoop ideal place to schedule finalizers?
No. Finalization should work for fully synchronous applications.
Another question about the division of responsibility between the .NET runtime and wit-bindgen
:
I'm currently working on adding a wasi:http
-based WasiHttpHandler
to System.Net.Http
, analogous to the existing BrowserHttpHandler
as a backing implementation of HttpClient
, per #98957. One of the things WasiHttpHandler
will need to do is turn a Pollable
into a Task
it can await
. Earlier, I proposed that WasiEventLoop
(to be provided by a wit-bindgen
helper library) would have a Register
function for this purpose, but we probably don't want WasiHttpHandler
to call that directly because it would make the .NET runtime (circularly) dependent on the wit-bindgen
helper library. What would be the best way to structure this?
The first thing that comes to mind would be to have the WASI implementation of ThreadPool
(EDIT: or some other class, since this isn't a ThreadPool
thing per se) include an internal
function for converting a Pollable
to a Task
and have it throw an exception until/unless wit-bindgen
generated code sets a handler for it. So wit-bindgen
-generated code would be responsible for calling both the SetEventLoopQueueFunction
we discussed above and a SetPollableRegisterFunction
. The upshot is that you wouldn't be able to use HttpClient
on WASI without wit-bindgen
being involved in some way, and that includes any tests we add to the .NET runtime, but I don't think there's any plausible way around that (other than writing bindings by hand, which I don't recommend).
I do expect we'll need to bundle wit-bindgen
-generated bindings as part of the .NET runtime for WASI so that WasiHttpHandler
can use them, as well as for testing. That doesn't mean the .NET runtime will depend on wit-bindgen
per se -- just that we'd need to generate the bindings, check them into the repo, and provide instructions and/or a script for regenerating the bindings as needed. For testing, we'd also need to bundle a copy of the helper library that implements WasiEventLoop
, but not ship it as part of the runtime.
As usual, I'm mostly thinking aloud here and am open to whatever the experts here think is best.
My primary concern is layering, i. e. the question of who provides the event loop implementation (
WasiEventLoop
above). My preference is for this to live underwit-bindgen
(not as generated code, necessarily, but as a supporting assembly/nuget, for example), at least for now. The main reason is that testing it would require the tool.
I probably don't fully follow. Testing would also require runtime in all cases, right ?
I'm currently working on adding a
wasi:http
-basedWasiHttpHandler
toSystem.Net.Http
I thought that you are going to use wit-bindgen
which will generate the code of the wasi:http
into the codebase of the runtime.
If we don't want it to be generated on every build of the runtime (and make dependency on rust/bindgen toolchain), we would commit the generated C# code into the runtime repo.
And I though that this generated code would contain the generated implementation of the WasiEventLoop
?
If not why not ?
And if not, perhaps runtime could have
internal interface IWasiEventLoop
{
void PollAllPollables();
Task RegisterPollable(Pollable pollable);
}
And static member on the WASI ThreadPool
into which the bindgen could install it via "unsafe accessors".
partial class ThreadPool
{
private static IWasiEventLoop? WasiEventLoop;
internal static void SetEventLoopQueueFunction(IWasiEventLoop wasiEventLoop)
}
If we don't want to include Pollable
type in the runtime, it could be just System.Object
for now.
Task RegisterPollable(object pollable)
Or even just the handle value
Task RegisterPollable(uint pollableHandle)
Well extra internal interface type IWasiEventLoop
is not making it easier either, so few registered delegates instead is fine too.
I'm currently working on adding a
wasi:http
-basedWasiHttpHandler
toSystem.Net.Http
I thought that you are going to use
wit-bindgen
which will generate the code of thewasi:http
into the codebase of the runtime. If we don't want it to be generated on every build of the runtime (and make dependency on rust/bindgen toolchain), we would commit the generated C# code into the runtime repo.
Yes, that's the plan. I'm writing WasiHttpHandler
itself by hand, making use of the generated bindings, which I plan to commit to the runtime repo with instructions and/or a script to regenerate them as needed.
And I though that this generated code would contain the generated implementation of the
WasiEventLoop
? If not why not ?
We were leaning towards putting reusable code like WasiEventLoop
, Result
, and Option
into a NuGet package that wit-bindgen
-generated code could refer to, which would reduce duplication in cases where an application depends on multiple packages which each use wit-bindgen
-generated code. But yes, the amount of reusable code is quite small, so we can just have wit-bindgen
"generate it" (i.e. copy it into the output) along with everything else.
WasiEventLoop
...wit-bindgen
"generate it" (i.e. copy it into the output) along with everything else.
Eventually that code belongs to runtime repo, I think. I still don't understand the preference to have it as part of wit-bindgen. @SingleAccretion could you please elaborate ?
could you please elaborate ?
As I said above:
The main reason is that testing it would require the tool.
The event loop must be generated at every entry point. The only entry point the runtime controls is Main
- inserting substantial event loop code into UCO methods would go against UCO design principles.
I therefore see async
-capable applications as a wit-bindgen
feature, since the pollable
marshaling is what actually enables true asynchrony.
Additionally, WasiEventLoop
is a design with an expiration date on it, which means we can't make it public in /runtime
. So even if it were in the runtime, wit-bindgen
would have to use private reflection to manipulate it - an odd composition.
This view is not set in stone. If we can come up with a layering that makes more sense given the constraints we have, let's make it happen.
Let's refocus this http
issue a bit. Say I am making bindings to APIs that return pollable
s from wasi:xyz
, to be published on nuget.org. How would I do that?
Additionally,
WasiEventLoop
is a design with an expiration date on it, which means we can't make it public in/runtime
. So even if it were in the runtime,wit-bindgen
would have to use private reflection to manipulate it - an odd composition.
Perhaps there's a middle ground: WasiEventLoop
lives in the runtime, but is private, and wit-bindgen
uses unsafe accessors to make use of it. And generates a static Task Register(Pollable)
function that application code can use, but that just defers to WasiEventLoop
's version.
Let's refocus this http issue a bit. Say I am making bindings to APIs that return pollables from wasi:xyz, to be published on nuget.org. How would I do that?
There's a precedent for having wit-bindgen
reuse an existing type instead of generating its own, so this could work one of two ways:
Pollable
, then wit-bindgen
would use that instead of generating its own versionPollable
in the .NET runtime (and I tend to agree we shouldn't since it's going away in WASIp3), then we could have an "official" WASIp2 nuget package that your wasi:xyz
package could depend on, and perhaps wit-bindgen
would just always generate code that depends on that WASIp2 package, ensuring everyone uses the same type.Perhaps there's a middle ground: WasiEventLoop lives in the runtime, but is private, and wit-bindgen uses unsafe accessors to make use of it.
That is what I was referring to - unsafe accessors are (fast) private reflection.
Unless we decide to consume WasiEventLoop
in the runtime, e. g. by saying that we would support async Main
'out of the box', I don't see the need for it to live in the runtime - nothing would use it (except tests, but tests can reference external packages just fine).
This is again mostly driven by the compatibility guarantees - once a public API is there in the runtime, it is there forever. If WasiEventLoop
was that "forever" model for WASI, I don't think we would hesitate to put it into the runtime, but we know that it is not.
If we decided not to include Pollable in the .NET runtime (and I tend to agree we shouldn't since it's going away in WASIp3), then we could have an "official" WASIp2 nuget package that your wasi:xyz package could depend on, and perhaps wit-bindgen would just always generate code that depends on that WASIp2 package, ensuring everyone uses the same type.
This is the sort of model I was thinking about. wasi:http
should use the same solution as wasi:xyz
.
Since we know WasiEventLoop
won't be public runtime API, we also know that this will be sort of required. Even if the physical implementation ends up living somewhere in this repository, the assembly providing access to it won't be part of the shared framework.
This is the sort of model I was thinking about.
wasi:http
should use the same solution aswasi:xyz
.
It certainly makes sense for a NuGet package that uses wasi:xyz
to depend on a WASIp2
package, but I'm adding support for wasi:http
to System.Net.Runtime
, which can't have any NuGet dependencies. So at a minimum we'd need to enable a "system" mode for wit-bindgen
to generate bindings suitable for use in this repo (i.e. do not require an external NuGet package). Those bindings would include a version of Pollable
for internal use, and we'd need to make sure both that Pollable
and the one in the WASIp2
NuGet package can be registered with WasiEventLoop
(probably by converting both to the underlying uint
handle, as @pavelsavara suggested).
Anyway, it sounds like the important point here is that we don't want to make any temporary APIs public in this repo, and we can be flexible about where each bit of code lives as long as we stick to that rule.
Thanks!
I'm adding support for
wasi:http
toSystem.Net.Runtime
, which can't have any NuGet dependencies.
I realized that some C# programs don't need to use HTTP and so the whole thing could be trimmed.
In which case we should be able to drop the runtime's dependency on wasi:http
and on WasiEventLoop
async
Main
'out of the box',
We always do async main in the browser for yielding reasons. I guess the similar reasons will come to WASI with p3.
If there is any import which uses pollable, it also means that there must be async main or async export.
In order to be able to do top level await in C#.
Even if we are able to implement it in p2 via blocking poll()
So, maybe always async main would simplify the WASI design too ?
I realized that some C# programs don't need to use HTTP and so the whole thing could be trimmed. In which case we should be able to drop the runtime's dependency on
wasi:http
and onWasiEventLoop
Most of it could be trimmed, yes. But at minimum we'll still need a way for wit-bindgen
-generated code to register a handler for ThreadPool
work items so that application code can use async
/await
even if it's not using System.Net.Http
.
Speaking of trimming: how can I tell the compiler not to trim an internal
function that's never called from within its assembly (e.g. to make it available to call externally via UnsafeAccessor
)?
Speaking of trimming: how can I tell the compiler not to trim an
internal
function that's never called from within its assembly (e.g. to make it available to call externally viaUnsafeAccessor
)?
You can add DynamicDependencyAttribute
onto something else that you know is not trimmed.
More in docs and runtime docs
You can also add xml config (but C# code is preferable). See https://github.com/dotnet/runtime/blob/f6dc71b611c6f725a626f52d3811ce6ad7e68ee1/src/mono/sample/wasm/browser-advanced/ILLink.Descriptors.xml#L1-L8
Also this seems related https://github.com/dotnet/runtime/issues/101195
Speaking of trimming: how can I tell the compiler not to trim an internal function that's never called from within its assembly (e.g. to make it available to call externally via UnsafeAccessor)?
Normally nothing special would be needed since trimming is performed on the whole final app, and it understands UnsafeAccessor
.
If there is any import which uses pollable, it also means that there must be async main or async export. In order to be able to do top level await in C#. Even if we are able to implement it in p2 via blocking poll()
Right.
So, maybe always async main would simplify the WASI design too ?
Very possible. It depends on what kind of app models WASI would be primarily targeting - library-like or application-like. We can't make library-likes work without code generation à la JSExport
.
Very possible. It depends on what kind of app models WASI would be primarily targeting - library-like or application-like. We can't make library-likes work without code generation à la
JSExport
.
I would say WASI (and the Component Model on which it is based) is primarily targeting library-like things going forward. wasi:cli/run
is one of many interfaces standardized as part of WASI, and it's the one which application-like things can export (while also exporting others, if desired). One can use wit-bindgen
to generate bindings for that interface and thereby create an application-like component without any help from the runtime. And the new componentize-dotnet project should make that easy by bundling all the needed tools together into a single package.
So I would say there's no urgent need for built-in async main support in the runtime; componentize-dotnet
will support targeting arbitrary WIT "worlds", wasi:cli
being just one example. I guess it comes down to whether developers will want to target WASI without using componentize-dotnet
or wit-bindgen
(and would presumably be okay with only targeting wasi:cli
and nothing else).
Here's what I have so far: https://github.com/dotnet/runtimelab/pull/2614
Note that I'm targeting the feature/NativeAOT-LLVM
branch of the runtimelab
repo in that PR since (AFAIK) that's the only branch capable of generating WASIp2 components. Happy to retarget it if that changes.
I've run some high-level, manual integration tests using https://github.com/dicej/dotnet9-wasi-http-example/tree/snapshot, and things are looking good there. Now I'm going to look at adding and/or reusing tests in the runtime repo and add the result to that PR.
Question: what should Task.Wait
do on WASIp2? Should it run a nested version of WasiEventLoop
? Currently it just busy waits indefinitely, which isn't great.
If I'm reading the code correctly, it looks like it's unsupported on the browser when multi-threading is disabled, so maybe we do the same for WASI? https://github.com/dotnet/runtimelab/blob/53342bee4b89a6c066d6eb3c7549028ca479c451/src/libraries/System.Private.CoreLib/src/System/Threading/ManualResetEventSlim.cs#L394-L397
If I'm reading the code correctly, it looks like it's unsupported on the browser when multi-threading is disabled, so maybe we do the same for WASI?
Yes, I think we need the same for single-threaded WASI.
Normally nothing special would be needed since trimming is performed on the whole final app, and it understands
UnsafeAccessor
.
The first pass of trimming is done (without final app) on the individual runtime assemblies separately. See https://github.com/dotnet/runtime/issues/106009
There is already ton of descriptors dealing with it in shared and in mono
I've been working to support
async
/await
in C# for doing concurrent I/O when targeting WASIp2. A key piece of that is thewasi:io/poll#poll
function, which accepts an arbitrary number ofpollable
handles representing e.g. socket readiness, HTTP protocol events, timer events, etc. It's analogous to the traditional POSIXpoll(2)
function. My goal is to provide aSystem.Threading.Tasks
-based abstraction on top ofwasi:io/poll
that supports idiomatic use ofasync
/await
, including the variousTask
combinators such asWhenAny
,WhenAll
,WhenEach
. I've done similar work for Python (using a customasyncio
event loop) and Rust (using a customasync
runtime), and am hoping to do the same for .NET.So far, I've managed to write a custom TaskScheduler which supports simple cases by maintaining a list of
Task
s and a list of thePollable
s those tasks areawait
ing. It has aRun
method which, in a loop, alternates between theTask
list and thePollable
list, executing the former and callingwasi:io/poll#poll
on the latter. That works well for simple cases.However, I've had trouble building more sophisticated examples using e.g. the new
Task.WhenEach
combinator due to the somewhat pervasive use ofThreadPool
as a deferred execution mechanism throughout theSystem.Threading.Tasks
library code. Given that WASI does not support multithreading and won't for the foreseeable future,ThreadPool
's methods currently throwSystem.PlatformNotSupportedException
, which makes it something of a landmine. For example, even though there's nothing inherently multithreaded aboutTask.WhenEach
, the current implementation relies on aManualResetValueTaskSourceCore
withRunContinuationsAsynchronously
set to true, which means it always queues continuations usingThreadPool.UnsafeQueueUserWorkItem
.Given that significant pieces of .NET's
async
/await
infrastructure currently relies on multithreading to function, I'm wondering what our options are for WASI. A few come to mind:async
/await
and use callbacks for concurrency in WASIp2. Besides being un-ergonomic, it would significantly restrict the set of both standard and third-party library features available for use on that platform, given that anything that deals with I/O is presumably either synchronous and multithreaded or asynchronous based onasync
/await
.async
/await
, but disallow use of features such asTask.WhenEach
which requireThreadPool
as an implementation detail, and possibly provide alternative implementations of those features which are single-thread-friendly.System.Threading.Tasks
which are not inherently multithreaded (but currently useThreadPool
) to support an alternative, single-threaded deferred execution mechanism on platforms that do not support multithreading.ThreadPool
implementation which simply queues work items without executing them, and provide some public API for running them from the main (and only) thread, e.g. as part of an application event loop.Thoughts?
I should note that I'm quite new to the .NET ecosystem, so I'm happy to be corrected if I've misunderstood anything.
See also https://github.com/dotnet/runtime/issues/98957, for which I would consider this issue a prerequisite.