google / marl

A hybrid thread / fiber task scheduler written in C++ 11
Apache License 2.0
1.85k stars 192 forks source link

Webassembly compatibility #236

Closed Guillaume227 closed 1 year ago

Guillaume227 commented 1 year ago

I attempted compiling with emscripten a project that depends on marl and I am hitting an error in osfiber_asm.h. marl/src/osfiber_asm.h:40:2: error: "Unsupported target"

Would anyone have any pointer regarding what it would take to port marl to webasm? Has it been considered/attempted?

AWoloszyn commented 1 year ago

My understanding of web assembly is that it does not yet have full fiber/stack switching support. Marl requires a working makecontext/swapcontext, or the ability to insert oyr item assembly version.

I think emscripten Asyncify might have enough support for marl, as marl does not move fibers between threads once started, but there needs to be some work there to verify and add the bits and pieces that are missing. Otherwise we have to wait until one of the various fiber/coroutine/async/stack switching proposals is finalized and available in webasm and emscripten.

Guillaume227 commented 1 year ago

Thank you for the informed reply. For my own information, where can I get more information and track those webasm/emscripten proposals? I am going through this now, trying to understand what bits might be missing for a marl port: https://emscripten.org/docs/api_reference/fiber.h.html

I do see a promising emscripten_fiber_swap

AWoloszyn commented 1 year ago

The different versions of this I have seen are in the following links:

https://github.com/WebAssembly/design/issues/1359 https://github.com/WebAssembly/stack-switching https://github.com/WebAssembly/stack-switching/blob/main/proposals/fibers/Explainer.md

One note about emscripten/ASYNCIFY is that IIUC it may make compilation time/program size balloon because it has to do a lot of analysis to figure out where fibers might get swapped out, but I am not completely understanding of all of the implications there.

Guillaume227 commented 1 year ago

Thanks! For the time being I managed to get something to run off standard (current version) emscripten, using emscripten/fiber.h and ASYNCIFY emscripten link option.

I just opened a PR: #238 I tested on firefox and chrome using the code in examples/ Those examples are small so I have not looked specifically at the output size / played with the optimization settings too much.

Here is what the tasks_in_tasks example looks like in the browser: http://localhost:8080/tasks_in_tasks.html image

ben-clayton commented 1 year ago

https://github.com/google/marl/pull/238 has been merged. Thank you very much for the contribution!

Are we good to close this issue?

Guillaume227 commented 1 year ago

Thanks for the review - all good from my end.