search

ocaml-multicore / domainslib

Parallel Programming over Domains
ISC License
172 stars 30 forks source link

sequential (OCaml 4) implementation of Task #95

Open gasche opened 1 year ago

gasche commented 1 year ago

Over at https://gitlab.com/gasche/domain-shims/-/issues/1, @Sudha247 expressed interest in relying on domain-shims to provide an OCaml 4 version of domainslib. But domain-shims, a pure-OCaml implementation of OCaml 5's Domain stdlib module, cannot support effect handlers, so we cannot support the current implementation of domainslib Task module.

I'm creating this issue to discuss the idea of supporting, in domainslib upstream, an alternate OCaml 4 implementation of the Task API. (Presumably, a task.ocaml4.ml module that would be switched instead of Task by some dune configuration magic.)

If there is interest, I would be happy to provide such an implementation (leaving the dune magic to others). I think it could be relatively simple (we are not trying to provide any actual parallelism) and low-effort to maintain.

Sudha247 commented 1 year ago

Thanks for opening the issue! Indeed, the idea was to have a sequential implementation of the Task module. There's a branch lying around for the same - https://github.com/ocaml-multicore/domainslib/tree/4.x. It strips Task of both Domain and effects, and merely retains a similar .mli interface. One issue with that branch is, it doesn't work with code that uses locking, for example channels (Chan) don't work in the current form.

Kate mentioned compiling with domain-shims could help domainslib work on OCaml 4, and I had a quick look at the library. Since the mock Domain interface runs on systhreads underneath, it could perhaps make the code with explicit synchronization (like Chan) work? We could also use domain-shims in the tests, and maybe even Task to make it work better with synchronisation constructs. Any thoughts?

My dune foo isn't great either, @kit-ty-kate suggested inserting appropriate enable_if flags for the OCaml 4 specific implementations. I think that's doable.

gasche commented 1 year ago

Yes, I believe that Chan would probably work indeed.

I tested domain-shims against all parallel tests in the compiler testsuite that don't use effect handlers. Everything worked out of the box, except one test (pingpong) that does busy-waiting. (Busy-waiting does not work if only one domain runs at a time and they are never preempted, it just loops forever.) My lesson is that most synchronization-using code probably works out of the box, but a few places may contain assumptions on the scheduler that don't work. (If you busy-wait, you need to call cpu_relax (which I implement as yield) or to release the domain lock; just having an allocation does not work, unlike "real" ocaml5 code where allocations act as preemption points.)