scope of domainslib

[c-cube]

Hi! Would domainslib be a good place to provide more concurrency building blocks, such as this?

• mpsc queues
• concurrent hashmap
• an equivalent of parking lot

[kayceesrk]

@c-cube I assume mpsc queues and hashmap are non-blocking data structures. A better place for these would be https://github.com/ocaml-multicore/lockfree/. Lock-free needs a big clean up, but here is my thought process between domainslib and lock-free.

Domainslib includes "blocking" data structures such as channels (which block domains) and tasks with async/await (with their notion of lightweight threads that block). Domainslib cannot be freely used in conjunction with other libraries which may include their own notions of blocking, such as eio. OTOH, lock-free only consists of non-blocking data structures and can be freely used by any library without worrying about the blocking semantics.

What is a parking lot? Does that include a notion of blocking, in which case, it doesn't belong to lock-free. We should think about where it would belong.

[c-cube]

parking lot is a C++ concurrency library used in webkit/chrome, with locks, queues, etc. :) An interesting link: https://webkit.org/blog/6161/locking-in-webkit/

it's also one of the most popular rust crates with the same inspiration.

Thanks for the link to lockfree, I think I forgot its existence. I wrote a queue in containers but it might be more naive than the ones already in lockfree…

[kayceesrk]

Thanks. @Sudha247 is going to clean up lock-free quickly, and seed it with the lock-free queue implementation from domainslib: https://github.com/ocaml-multicore/domainslib/blob/master/lib/ws_deque.ml. We may then be in a position to accept PRs.

[Sudha247]

Hey @c-cube, I've cleaned up lockfree, any PRs for adding non-blocking data structures appreciated! (https://github.com/ocaml-multicore/lockfree)

[c-cube]

Good to know :slightly_smiling_face: . The list above mostly contains blocking structures, but at least a deque is useful for schedulers!

[kayceesrk]

Closing this issue as Saturn will be the place for all parallel data structures. Moreover, domain-local await will allow domainslib to utilise blocking data structures from Saturn.