Are there any benchmarks?

[d4h0]

Hi,

Stakker looks fascinating!

I'm wondering if you have planned to publish benchmarks that compare Stakker with other options.

I think, that could help with the marketing of the crate (if the results are good... 😄).

Also: Is my assumption correct, that basically everything has to be re-written for Stakker (HTTP servers/clients, etc.)? Or would it be easy to reuse crates that are written on top of "pure" Mio, for example?

[uazu]

Yes, I did some benchmarking but it isn't complete enough to publish yet. I tested doing a calculation that stresses both data and instruction cache and then passing the resulting value to the next actor, and so on around a ring of actors. The number of actors and the length of the data-part and instruction-part of the calculation could be varied. Also the number of rings running in parallel could be varied. I tested Stakker against a solution based on Rc<RefCell> and also actors running in separate threads connected with crossbeam channels. The conclusions so far are:

• Stakker does very well, roughly comparable to calling another actor through Rc<RefCell> once you've added code to handle re-entrant calls to Rc<RefCell>-based actors.
• Crossbeam+threads does not work well unless you saturate it with work, which is why I had to add a number of rings running in parallel to the tests. If for a moment there is no work, the kernel deschedules the thread and then there is a delay to get that thread running again.
• For small units of work, less than a few hundred lines of generated assembler, single-thread Stakker easily wins against saturated multi-thread crossbeam. Bigger than that, crossbeam first draws even, and then starts to get ahead (so long as it stays saturated with enough work).

So this backs up what I claimed in the blog post. However trying to nail down theses figures more precisely gets difficult, as there is variation up/down with small changes to the code, I guess due to cache effects. I would need to do a lot of runs and plot them to see the trend. Also it would be better to also test against other channel implementations such as flume and the channel implementation inside Tokio.

Regarding interfacing with the existing ecosystem, there is no ready-made "glue" available as yet. If a crate was written to just do protocol and not talk directly to an I/O layer (i.e. not hardcoded to one solution) then interfacing to it should be relatively easy. (It's possible that some crates even if coded for Tokio for example may also provide another interface which just does protocol, which we could interface to.) If something is coded to talk directly to mio then that's probably just as unhelpful as it being hardcoded to talk to anything else, although it depends on the exact way they interface (maybe they could be adapted more easily).

It may be possible someday to support running external async/await code, i.e. act as an executor. Then maybe we can pull in some of the ecosystem that way. It needs investigating. Otherwise you can run an existing executor (Tokio/async_std/etc) in another thread and communicate to the Stakker thread with channels or something. Whether this is efficient enough depends on how central that aspect is to your application.

If you or anyone else has better ideas about how to (with relatively small effort) bring in some part of the ecosystem, then I'd be interested to hear.