interest in improved scheduler

[maddanio]

Hello,

First off: thank you for putting asio out there. Gives at least me a simple and very good goto solution for efficient and easy to maintain networking code. Recently I have been hitting performance boundaries. We are dealing with very low-power (in terms of cpu) networked cameras that very quickly lose frames if not picket up in time. Since we are running hundreds of these keeping this up is not easy. While mulling over ways to get better at it, I stumbled (again) across sean parent's great talk on concurrency: here and then looked at the current implementation of the core scheduler in asio. it seems it is of the simplest design, i.e. single-queue single-lock. so i was wondering if there is any interest in upgrading it to what sean showed in the talk to be relatively feasible to implement and very close to optimal: a multi-queue work-stealing scheduler. as far as i understand it means re-implementing the scheduler interface, so relatively contained. if there is interest in this i might start on this some time. I think though for this to succeed it would need some qualified reviewers and ideally interest of mr kohlhoff himself.

[mabrarov]

I was thinking about utilizing lock-free queue for asio::io_context::strand class which current implementation uses a pool of mutexes. Unfortunately, lock-free queue is LIFO which may give unexpected results for the users of asio::io_context::strand. Well, asio::io_context::strand doesn't guarantee ordering, but LIFO is still something unexpected which may lead to unwanted behavior under heavy usage of asio::io_context::strand.

[maddanio]

I think the low hanging fruit would be work stealing. It would make extensive use of try_lock which afaik usually uses only atomics anyhow. Actual locking is mostly just for waiting for new work.

[kashirin-alex]

Might be useful, anyhow, Are you using lambda for handlers?

• lambda might/is (with size) require another allocator for the Closure storage (pointer is used for the queue in this case) (subject of the dynamic-memory of queue's item allocation)

as example. https://github.com/kashirin-alex/swc-db/blob/6298182fa9f4d8079ac245fc90578bc689086a8c/src/cc/include/swcdb/core/comm/SerializedClient.h#L94 async-request is at the https://github.com/kashirin-alex/swc-db/blob/6298182fa9f4d8079ac245fc90578bc689086a8c/src/cc/include/swcdb/core/comm/SerializedClient.h#L127

-- if you do make a change to a non-lambda handlers, It will be interesting to know if there is a reasonable difference.