search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.04k stars 4.68k forks source link

[browser][MT] Multithreading and JavaScript async interop in .NET 9 #85592

Closed lambdageek closed 4 months ago

lambdageek commented 1 year ago

Tracking issue for further work on JS interop and multithreading.

Constituent part of https://github.com/dotnet/runtime/issues/76956

Goals

Lower priority goals

Non-goals

Design discussion and experiment

Depending on selected design

Bugs

Memory growth, alignment

Broken tests

Nice to have

Progress

Future

lambdageek commented 1 year ago

/cc @lewing @pavelsavara

ghost commented 1 year ago

Tagging subscribers to 'arch-wasm': @lewing See info in area-owners.md if you want to be subscribed.

Issue Details
Tracking issue for further work on JS interop and multithreading. Constituent part of https://github.com/dotnet/runtime/issues/76956 ### Scenarios * [ ] *[blazor]* Single threaded Blazor WebAssembly code can turn on threading and continue working. Main thread interop keeps working. "Hidden" async interop in the HTTP stack and in aspnetcore keeps working. * [x] *[pool-promise]* Multi-threaded WebAssembly code can do async JS interop from threadpool threads using promises https://github.com/dotnet/runtime/issues/84489 * [ ] *[pool-timeout]* Multi-threaded WebAssembly code can do async JS interop from threadpool threads using timeouts and event listeners * [ ] *[thread-eventloop]* Multi-threaded WebAssembly code can start new threads using `new Thread` and use async interop ### Work Items #### *[blazor]* * [ ] Async runtime code that uses `ConfigureAwait(false)` behaves correctly when using async JS interop (e.g. HTTP stack) * [ ] The aspnetcore `BeginInvokeDotNet`/`EndInvokeDotNetAfterTask` APIs work correctly in multithreaded apps (`ContinueWith(..., TaskScheduler.Default)`). #### *[pool-promise]* * [x] async Tasks started with `Task.Run` that are running off the main thread can use JS interop https://github.com/dotnet/runtime/issues/84489 * [x] awaiting unsettled JS interop promises keep threadpool threads alive https://github.com/dotnet/runtime/issues/84489 * [ ] When a thread shuts down, any live c# object proxies in the finalization registry are released and the finalization registry entries are disarmed. (ie prevent crashes when the finalization registry tries to run c# code while the webworker is no longer attached). (Part of a plan here: https://github.com/dotnet/runtime/pull/84494#issuecomment-1517710452) #### *[pool-timeout]* * [ ] Reachable JS objects (from `setTimeout`, event handlers, other non-promise based async interop) that reference C# objects keep the threadpool thread alive. https://github.com/dotnet/runtime/issues/85052 #### *[thread-eventloop]* * [ ] **[API proposal]** `new Thread` has a way to spin up "external eventloop" threads * [ ] **[API proposal]** Give .NET code a way to keep an external eventloop thread alive ("keepalive token") * [ ] **JS API** provide a way for JS code to keep a .NET external eventloop thread alive * [ ] awaiting unsettled JS interop promises keeps external eventloop threads alive * [ ] reachable JS objects that reference C# objects keep an external eventloop thread alive * [ ] `SynchronizationContext` for external eventloop threads that allows posting work to the thread from other C# threads (ie extend `JSSynchronizationContext` to non-main threads)
Author: lambdageek
Assignees: lambdageek
Labels: `arch-wasm`, `tracking`, `area-System.Runtime.InteropServices.JavaScript`
Milestone: 8.0.0
pavelsavara commented 1 year ago

Blazor's requirements are:

1) Only one work item runs at at time on this sync context 2) You can post to it from any thread 3) Once on the sync context, JS interop must work (so presumably means we can be sure we're now on the main thread)

Rippletank commented 10 months ago

I'm using the latest .net8.0 RC2 release in a multithreaded WASM app using UNO and Skiasharp. I am getting the "Please use dedicated worker...etc" from the AssertWebWorkerContext() method as expected.

The work around is sending JSInterop calls to the main thread, which works but results in some animation stuttering particularly with large db/cryptography jobs.

I would like to try out the WebWorker class but it does not appear to be included in the Microsoft.NETCore.App\8.0.0-rc.2.23479.6\System.Runtime.InteropServices.JavaScript.dll which the application is using and because of the internal methods I see how to implement it outside of the dotnet runtime.

Is this just not available in released packages yet or is there some way to enable it?

I notice there is a sample that uses it but doesn't this need compiling the runtime from source?

pavelsavara commented 10 months ago

I'm using the latest .net8.0 RC2 release in a multithreaded WASM app using UNO and Skiasharp.

@Rippletank Threads are not supported in .Net 8. As you can see above, there are many reasons why we are not ready for it.

jtorjo commented 10 months ago

@pavelsavara Thanks for the update. I'm curious if we'll ever have this, because if I remember correctly, this was .net7 --> .net8, and now it's .net8 --> .net9

Rippletank commented 10 months ago

@Rippletank Threads are not supported in .Net 8. As you can see above, there are many reasons why we are not ready for it.

Ok, I see. It is confusing because multithreading is enabled with Uno web assembly apps and it works within the app itself but obviously doesn't work with the interop.

I'm curious, is ok to use it, apart from interop issues, or is it generally problematic at the moment? For things like http etc.

pavelsavara commented 10 months ago

I'm curious, is ok to use it, apart from interop issues, or is it generally problematic at the moment? For things like http etc.

MT is problematic in Net8 in my experience, that's why it's experimental. I suggest you ask Uno how to turn it of.

@pavelsavara Thanks for the update. I'm curious if we'll ever have this, because if I remember correctly, this was .net7 --> .net8, and now it's .net8 --> .net9

Yes those are not easy problems. We are not out of the woods yet for Net9 either. We don't want to ship product which could randomly deadlock. Wish us luck.

jtorjo commented 10 months ago

Yes those are not easy problems. We are not out of the woods yet for Net9 either. We don't want to ship product which could randomly deadlock. Wish us luck.

@pavelsavara I'm definitely wishing you luck. But it seems we won't have it for another 2+ years, if I understand you correctly.

pavelsavara commented 10 months ago

@pavelsavara I'm definitely wishing you luck. But it seems we won't have it for another 2+ years, if I understand you correctly.

I understand that you are waiting with some specific use-case for this ? Could you please share more details why/how you would benefit from threads in browser ? Also anyone else reading this, I would like to hear about your use-cases.

jtorjo commented 10 months ago

@pavelsavara The company I work for migrates Silverlight apps to webassembly.

Thus, we end up going from apps that were written using multiple threads (Silverlight), to a single thread (webassembly). Most of the time, this is not an issue, but every now we end up with some bottlenecks that are insanely hard to fix (time-consuming parts of the code will end up being executed, freezing the UI).

That's where multi-threading would help.

ivanjx commented 10 months ago

@pavelsavara i would like to do image processing (like qr/barcode scanning) in separate threads.

Rippletank commented 10 months ago

Similar case here. Our educational app was built in Xamarin, now Maui but really it's a SkiaSharp app with a few support services supplied by the host. Getting it running as an Uno WebAssembly app was very quick but its taken much much longer to get it running close to acceptably smooth.

The WebAssembly and PWA installs, will be great for teachers working inside school networks. But it needs to keep as much of the app feel as possible.

The multithreading in Uno seems to use multiple dotnet.native.workers. It hasn't given any obvious problems yet, outside of interop yet, but it's still being tested. Unfortunately, many of the background tasks involve database access which does require interop. Add to that cryptography because it's not implemented yet on the .net side.

So basically, I was looking for a way for C# code to call the db/crypto functions (webworker safe APIs) transparently from whichever worker instance it finds itself in, except for maybe calling ImportAsync if that worker was not set up. But the only way is to pass them to the main thread.

Fydar commented 10 months ago

I'm interested in seeing what the Blazor team can do with multithreading internally. I'm using client-side Blazor components embedded into my server-rendered application; but when the UI being unresponsive for the first second that I'm loading the page is incredibly disruptive in the user flow.

With threading, I suspect the unresponsive UI could be fixed.

I would love to even allow a Blazor runtime to be executed in a web worker to allow for sharing state between multiple browser tabs so that startup costs are only encountered once.

pavel-zhur commented 8 months ago

@pavelsavara I'm definitely wishing you luck. But it seems we won't have it for another 2+ years, if I understand you correctly.

I understand that you are waiting with some specific use-case for this ? Could you please share more details why/how you would benefit from threads in browser ? Also anyone else reading this, I would like to hear about your use-cases.

In my case, in the app that I'm building, the UI freezes while any time consuming operation is taking place (like deserialization of 10MB of json, or interoping with JS - for instance, storing data in the local cache, which also includes serialization).

As a result, while the app is busy, it is unable to immediately react to the user interaction.

It seems I need to make blazor process only extremely small pieces of data or to offload heavy data processing to other libraries that use web workers. I wish it all just worked natively and blazor could utilize multiple cores of the client browser out of the box.

rootflood commented 8 months ago

@pavelsavara I'm definitely wishing you luck. But it seems we won't have it for another 2+ years, if I understand you correctly.

I understand that you are waiting with some specific use-case for this ? Could you please share more details why/how you would benefit from threads in browser ? Also anyone else reading this, I would like to hear about your use-cases.

Task.Wait() method in WASM is blocked without threading in some of major cases end of some frameworks there is wait method and if task.Wait don't work all of that code should change to async

pavelsavara commented 8 months ago

Task.Wait() method in WASM is blocked

We plan to keep it throwing PNSE on the UI thread (and any thread with JS interop). Reasons is that when the thread is synchronously blocked or spin-blocked, it could not serve the browser event loop. 1) handle any UI event or rendering 2) handle any networking event (HTTP and WS) 3) new threads can't be created 4) browser dev tools are not working properly. VS debugger is probably hanging too.

and any of above could lead to deadlocks.

Task.Wait() would start working on any new Thread or Task.Run in the thread pool. So you could move your synchronously blocking code out of UI thread and rest should be fine.

Is that acceptable trade-off ? If not, why not ? What scenarios would be prevented by moving to thread pool ? What would be better design option ?

Matheos96 commented 8 months ago

@pavelsavara I'm definitely wishing you luck. But it seems we won't have it for another 2+ years, if I understand you correctly.

I understand that you are waiting with some specific use-case for this ? Could you please share more details why/how you would benefit from threads in browser ? Also anyone else reading this, I would like to hear about your use-cases.

Here is our use-case:

We have a Client-side only Blazor WASM app. Our main component is a PCB 3D Board Viewer. We load the data from our database but the data itself is not that complex (source from CAD formats). It does not technically contain the 3D data we need to render. So we build the 3D models on demand, based on our own Data model which is being deserialized and interpreted, all client side. The amount of details on such a board is immense, meaning we will have to do A LOT of calculations. For big boards we may have waiting times for more than 4 minutes, simply waiting for the deserialization (not too bad) and generation of 3D vertices, uvs and so on. During this time, the browser is pretty much completely frozen, we have some Task.Delay(1)'s here and there to update a progress bar in between but often we get the "not responding" message anyway. This is obviously a horrible user experience. Our dream is to be able to offload this generation to separate threads, for example separate board layers don't depend on each other and can be built completely in parallel. Ideally we want to show something really quickly, and then in the background keep calculating stuff that become available once done. This is obviously nothing but a dream at this point, still being restricted to the UI thread (and potentially web workers which we will look into). The whole thing does not get any better from the fact that we depend on A LOT of JS interop too... But that is another story.

So that is our use-case. One could ask why we don't do server-side rendering, but the fact is that we don't really have a deployed production "web" version yet. Currently we are providing this as an extension of our legacy phat client (embedding the SPA in a CEF Sharp window). One of our selling points is also the fact that we "make everything happen" in the browser, without the need for a backend. We also offer a "from file" option which keeps all data in the browser, meaning we definately have to do the calculations in the wasm app.

curiousdannii commented 8 months ago

Copying my use case from #68162:

I have an app I'm trying to port, which calls a blocking function in a non-managed dll:

[DllImport("Glk")]
internal static extern void glk_select(ref Event ev);

To port it to WASM/JS it instead needs to call an async JS function (ignore that it no longer takes any arguments):

[JSImport("glk_select", "main.js")]
internal static partial Task glk_select();

I've made an interface that lets me link to either the DLL or the JSImport, but I just need some way to block while I wait for the promise to resolve. And that's pretty much it! I don't really care how this is accomplished. While the details of which threads run where will matter for other people and their use cases, for me it's just an implementation detail. An elegant solution is more what I'm after.

(I also tried making the app async, but it's a large messy legacy app (28k+ LOC) and I would end up needing convert nearly everything to async functions, so threads seems a better option.) (And async functions not having ref or out arguments and VB.net not having tuple destructuring makes it even grosser to try to convert.)

iSeiryu commented 8 months ago

Copy-pasting use-cases from here: https://www.reddit.com/r/Blazor/comments/199o6e6/why_is_multithreading_such_a_requested_feature/

pavelsavara commented 8 months ago

You hope to dear god you can have something like promises that you get in JS

I'm not sure I follow, could you please elaborate @iSeiryu ? You can marshal Task/Promise with JSImport.

iSeiryu commented 8 months ago

@pavelsavara those are not my words. That's just a list of use-cases I came across yesterday. Btw, someone else in that Reddit thread asked a similar question.

dennis-garavsky commented 8 months ago

enable blocking Task.Wait and lock() like APIs from C# user code on all threads

We will also benefit from this enhancement at DevExpress in certain products, and our customers will for sure benefit in their projects (while migrating shared code to .NET 8 and Blazor).

For instance, in our application framework DevExpress XAF (powered by ASP.NET Core Blazor), we researched ways to plug in a middle tier service into our existing code base with Blazor WebAssembly and EF Core. The following code throws "Cannot wait on monitors on this runtime" with Blazor .NET 8 as it did 5 years ago when we started supporting Blazor in XAF and experimented with WebAssembly back then (dropped WebAssembly due to low performance and the lack of multi-threading support). Today we still rely on Blazor Server in XAF Blazor, but want to support WebAssembly render modes in the future.

public class WebApiSecuredDataServerClient
        protected override TResult Invoke<TResult>(string action) {
            return StaSafeHelper.Invoke(() => InvokeAsync<TResult>(action, CancellationToken.None).ConfigureAwait(false).GetAwaiter().GetResult());
        }

NOTE: our Blazor UI Components already fully support both WebAssembly and Server modes from November 2023.

As other readers also noted, we also want to reuse a large amount of our existing framework core code in Blazor WebAssembly, as we successfully did in Blazor Server, WinForms and WebForms for over 15 years now. Converting "nearly everything to async functions" in the framework core code is NOT realistic due to the huge breaking changes for other customers and rewrite costs, of course (it may be easier to drop everything and start over with a new product then).

Thank you for your consideration.

pavelsavara commented 8 months ago

enable blocking Task.Wait and lock() like APIs from C# user code on all threads

We are still re-considering this one. See my other comment.

Converting "nearly everything to async functions" in the framework core code is NOT realistic

We understand that ^^.

Alternatives are

curiousdannii commented 7 months ago

The mono runtime is compiled using Emscripten. Is this also the case with AOT compilation? If that's the case, any chance that we could run ASYNCIFY like we would for any other Emscripten app? That could be another way of converting sync functions into async.

glutio commented 6 months ago

Trying .net9 preview 2, wasmbrowser template, the all synchronous call .net->js->.net hangs the browser window. Seems like a regression from .net8

pavelsavara commented 6 months ago

Trying .net9 preview 2, wasmbrowser template, the all synchronous call .net->js->.net hangs the browser window. Seems like a regression from .net8

This is by design on multi-threaded build. The reason is that managed code is not running on UI thread, but your JS is running there. The managed thread is blocked waiting for the first call to return from JS while you try to send it another message with managed call.

Nested synchronous JS interop calls would not be supported in MT at all. It will throw PNSE in next previews. Non-nested synchronous JS interop would be possible in .Net->JS direction. Non-nested synchronous JS->.Net direction only behind a configuration flag and could lead to deadlocks in some scenarios.

Here is work in progress PR on the topic. https://github.com/dotnet/runtime/pull/99833

Single-threaded build would not change in this regard.

maxkatz6 commented 5 months ago

@pavelsavara would it be possible to call a sync .NET exported function (or a JSImport callback) without blocking JS thread? I.e. discarding a result. Making .NET function async is a decent workaround here, but I don't see why it should be forced for "fire and forget" scenarios. Thanks.

maxkatz6 commented 5 months ago

Also, will new behavior be toggleable? It's not yet clear if we can support new threading model in .NET 9 WASM, since it has to work with sync requestAnimationFrame callbacks, webGL and Skia on top of Emscripten, something that yet needs to be tested. But .NET 8 WASM threading model might be sufficient for most Avalonia apps (although we would love to make new one supported too).

Or even better, could it be possibly to initialize .NET thread on a physical UI thread? In our case it would work as a render thread with Skia and WebGL. While the rest of the app is managed on a normal deputy thread.

kekekeks commented 5 months ago

I believe that the lack of access to the "true" UI thread would completely break WebGL interop. WebGL doesn't have a dedicated present/swapBuffers call and instead just presents render results once the user code exits the current event callback. If JS API calls are getting enqueued from a worker thread, that would mean that webgl content will be "presented" on every single call, thus making any kind of rendering impossible.

Another problem would be WebGL usage with SkiaSharp: Skia uses WebGL internally and expects those calls to happen on the "true" UI thread rather than on a web worker.

pavelsavara commented 5 months ago

"fire and forget" scenarios.

I created DiscardNoWait for it and it's applied to void methods. But it's not on public API, yet. It executes asynchronously similar to methods returning Task, but it doesn't need to marshal the Task. If you search this repo you can see it used in some tests.

You can help me by pushing it thru API review process (and have discussion about it's name on that PR).

https://github.com/dotnet/runtime/blob/5111fdc0dc464f01647d6b6078342f451bf3a499/src/libraries/System.Runtime.InteropServices.JavaScript/src/System/Runtime/InteropServices/JavaScript/JSMarshalerType.cs#L47-L54

Could it be possibly to initialize .NET thread on a physical UI thread?

We will not support running managed code on UI thread, I'm confident now. It leads to too many deadlocks which are not possible to predict or even test against.

At the moment the UI thread is still attached to Mono VM, but that's going to change as soon as possible.

Also, will new behavior be toggleable?

There is toggle for allowing synchronous JSExport (with all the consequences of spin-blocking UI thread) We just have to make it into MSBuild property It's quite easy to cause deadlock with it, but not so bad as running managed code on UI thread.

But .NET 8 WASM threading model might be sufficient for most Avalonia apps

Net8 WASM threading is very broken, my 2c: don't do it to your users.

I believe that the lack of access to the "true" UI thread would completely break WebGL interop.

I would like to learn more, how to make more native scenarios possible with MT. This is not right place for detailed discussion, so I made https://github.com/dotnet/runtime/issues/101421 for it

maxkatz6 commented 5 months ago

You can help me by pushing it thru API review process

How can I help with the review process? Naming looks fine, easier to understand than "OneWay".

pavelsavara commented 5 months ago

You can help me by pushing it thru API review process

How can I help with the review process? Naming looks fine, easier to understand than "OneWay".

You can follow https://github.com/dotnet/runtime/blob/main/docs/project/api-review-process.md Create issue with proposal coach it thru the API review meeting. Also create PR which removes the API from https://github.com/dotnet/runtime/blob/bc9fc5a774d96f95abe0ea5c90fac48b38ed2e67/src/libraries/System.Runtime.InteropServices.JavaScript/src/CompatibilitySuppressions.xml#L15-L20

Which will make it public.

Examples are https://github.com/dotnet/runtime/issues?q=is%3Aissue+label%3Aapi-approved+is%3Aclosed

@maxkatz6

kekekeks commented 5 months ago

Will async callbacks be only supported for JSExport? We don't use those and are using [JSMarshalAs(JSType.Function)].

If we try to change JS->.NET callbacks to return tasks, the SDK complains about 18>TimerHelper.cs(14,113): Error SYSLIB1072 : The type 'System.Func<System.Threading.Tasks.Task>' is not supported by source-generated JavaScript interop. The generated source will not handle marshalling of parameter 'callback'. For more information see https://aka.ms/dotnet-wasm-jsinterop (https://learn.microsoft.com/dotnet/fundamentals/syslib-diagnostics/syslib1072)

What is the new intended way to pass delegates to JS side?

pavelsavara commented 4 months ago

The type 'System.Func' is not supported

That's a (known) gap. Could you please open separate issue for it and ping me there ? @kekekeks

pavelsavara commented 4 months ago

I'm closing this as complete for the scope of Net9 for the dotnet runtime.

Summary

The difference between Net8 and Net9 threading is that it actually works now. 😅 All the managed code is running in the web workers and that makes it possible to use blocking .Wait on all managed threads. Limitation (by design) is that you can only call asynchronous C# methods as JSExport from the UI thread (main JS with DOM). All JavaScript interop via JSImport is dispatched on to UI thread via low level async message. Another limitation by design is that you can't make nested synchronous callback from inside synchronous JSImport call. HTTP and WebSocket connections are made from UI thread on your behalf.

Status is still: experimental

Blazor

For Blazor WASM, the multi-threading feature was cut for Net9. The biggest gaps in Blazor are described in https://github.com/dotnet/aspnetcore/issues/54365

Demo

I updated the Raytracer demo https://pavelsavara.github.io/dotnet-wasm-raytracer/

Performance

We didn't optimize for multi-threaded performance yet. As compared to single-threaded dotnet on WASM, it will be slightly slower per core, but you can use many cores now. It should also allow you to offload CPU intensive work from the main thread and keep the DOM responsive.

Future work

There is draft of JSWebWorker API which we will explore in the future. It can allow you to interact with another JS than the UI thread. We didn't implement any of the sync-over-async scenarios in this release.

If there are bugs or missing features, please create new issue on runtime repo and ping me, we will triage and prioritize.

curiousdannii commented 4 months ago

@pavelsavara Is the working threading in the Net9 preview 3 from April 11? Or can we expect it in the next preview?

pavelsavara commented 4 months ago

The demo above is on Preview 3. Net9 Preview 3 has most of it and Preview 4 has fixes and some cleanup, I expect that we will improve quality further. If you run into issues you can always try nightly build and see if it was already fixed.

kekekeks commented 4 months ago

Is any kind of debugging supposed to work with MT mode? I can't get vscode nor /_framework/debug to stop on breakpoints.

pavelsavara commented 4 months ago

Is any kind of debugging supposed to work with MT mode? I can't get vscode nor /_framework/debug to stop on breakpoints.

There is open issue for that https://github.com/dotnet/runtime/issues/81282 and few more for broken tests