search

mratsim / weave

A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead
Other
533 stars 22 forks source link

Distributed computing #73

Open mratsim opened 4 years ago

mratsim commented 4 years ago

Naive distributed computing requires the following things:

Thanks to our message-passing based design, we should be able to reuse large part of the code though some hierarchical work-stealing probably needs to be introduced.

See Thesis for MPI based channels

image

Further distributed channel alternatives could be ZeroMQ, see presentation: http://irpf90.ups-tlse.fr/files/oslo_zmq.pdf and Nanomsg which is MIT-licensed https://github.com/nanomsg/nng see writeup: https://nanomsg.github.io/nng/RATIONALE.html

mratsim commented 4 years ago

Very interesting paper to hide latencies in distributed work-stealing: http://reports-archive.adm.cs.cmu.edu/anon/2016/CMU-CS-16-112R.pdf

This might also help with IO latencies (#22 )

mratsim commented 4 years ago

For distributed computing, the SparseSet introduced in https://github.com/mratsim/weave/pull/15 is too memory hungry.

It can be replaced by succinct data structure. Constant-time rank support for succinct data structure (which requires a bit of overhead over the bitset) is the subject of in-depth research. image