search

stlab / libraries

ASL libraries will be migrated here in the stlab namespace, new libraries will be created here.
https://stlab.cc
Boost Software License 1.0
662 stars 66 forks source link

How does your Concurrency library compare to, e.g., Boost.Fiber? #112

Closed niekbouman closed 6 years ago

niekbouman commented 6 years ago

Dear developers,

I am working on a software project (not yet visible on github) in "Secure multiparty computation", which is a privacy-preserving computation paradigm from the field of cryptography, in which many basic operations like multiplication of numbers require interaction between several parties (connected via an IP network). The problem above maps well to an async computation-graph type of solution, which is the concurrency pattern that your stlab.Concurrency library seems to provide.

I have started my project using Boost.Fiber (from Oliver Kowalke), which also integrates fairly well with Boost.ASIO. Now I just saw your announcement of stlab.Concurrency, which looks very interesting (in particular the "one-to-many"-future feature, or in other words the "easily create splits" feature), and seems to aim to achieve similar goals as Boost.Fiber.

Would you be willing to provide some more information for library-users that would help in judging the benefits of your [Concurrency] library over modern c++ alternatives,[like boost.fiber]? (BTW, Asio's author, Kohlhoff, is also working on a new executors library, not sure how much his goals overlap with yours...)

Also in terms of benchmarking, Alexander Temerev has a benchmark called skynet, it would be interesting to see how stlab's Concurrency library compares there to the state of the art. https://github.com/atemerev/skynet

Anyway, thank you for making your work available online! kind regards, niek

sean-parent commented 6 years ago

Hi Niek, Thanks for the query - Fibers are a cooperatively scheduled thread-like system and require access to the execution context through Boost.Context. In general, fibers are for managing lightweight concurrency within a single thread, however, because of the requirement of accessing the execution context they are not feasible to run in some environments (like within any of the web environments, wasm, asm.js, or PNaCl).

Channels are a way for coroutine-like processes to communicate within a tasking system (they may or may not be running on separate threads, depending on the tasking system used). stlab concurrency provides a portable, high performance, task stealing scheduler as well as executor to support both Apple's libdispatch (also available on may linux systems) and Windows thread pools. Because of the potential for running on separate threads, there is some inherent overhead in channels and they are intended for fairly coarse tasks. Channels are modeled after channels in the Go language and we plan to support the TS coroutines at some point (I'd like to get that into our next release). Channels were originally written so that I could scale an image processing pipeline to run within a single threaded environment (asm.js) while scaling up to a many-core desktop machine.

From the short description of your problem, you might also look at Intel TBB flow graphs. They provide similar capabilities to channels, but have some advantages if you know the structure of your graph up front.

You might also look at HPX which is designed to build concurrent systems that scale to large grids (and down as small as a Raspberry Pie).

Thanks for sending the benchmark - I agree that we should attempt to run it but at this point I don't think we would perform well. Although we are working to reduce overhead (we made some progress on the 1.1 release - it hasn't been a major focus. That said, the tasking system itself that we provide I have benchmarked and we do very well against every task stealing system I've put it against.

FelixPetriconi commented 6 years ago

Closed as not a bug.