Swarm and its downstream proto libraries are all currently tightly tied together with ocean's EpollSelectDispatcher. The bulk of the direct usage is in swarm itself (the proto libraries just pass epoll instances into swarm internals, or occasionally use other ocean epoll-based functionality like timers), but the neo protos also make a fair bit of use of the epoll-based Task framework.
The major consequence here is that swarm and the protos are impossible to use with other popular D2 task-scheduling libraries such as vibe.d, to say nothing of the possibility of integration with other languages, where async IO may be managed with other mechanisms and other eventloop controllers.
Given that in practice all the different use-cases will likely be using epoll underneath the hood, it ought to be possible to achieve better integration than this. The question is what the most appropriate mechanism.
Based on a very provisional glance through the codebase there appear to be 3 main use-cases within swarm neo:
Task.getThis() calls to get the current task instance (and yield etc)
EpollSelectDispatcher.register(...) calls to register listeners and timers
ISelectClient, mostly through the ISelectListener subclass
Legacy swarm also includes some unregister and setExpiryRegistration calls, while the protos (without checking for neo vs. legacy) similarly seem to use just {un}register and Task.getThis() calls.
This all feels high-level enough that it ought to be possible to abstract it away from EpollSelectDispatcher specifics, but that begs some questions:
is it better to make create an abstraction layer for swarm's specific needs, or to rework ocean to support different options?
if we assume that epoll is the underlying mechanism, can we set things up so that e.g. epoll events can be (un)registered via EpollSelectDispatcher while the eventloop can be run by another library (or vice versa)?
how (if at all) can we best integrate with languages other than D?
Swarm and its downstream proto libraries are all currently tightly tied together with ocean's
EpollSelectDispatcher. The bulk of the direct usage is in swarm itself (the proto libraries just pass epoll instances into swarm internals, or occasionally use other ocean epoll-based functionality like timers), but the neo protos also make a fair bit of use of the epoll-basedTaskframework.The major consequence here is that swarm and the protos are impossible to use with other popular D2 task-scheduling libraries such as vibe.d, to say nothing of the possibility of integration with other languages, where async IO may be managed with other mechanisms and other eventloop controllers.
Given that in practice all the different use-cases will likely be using epoll underneath the hood, it ought to be possible to achieve better integration than this. The question is what the most appropriate mechanism.
Based on a very provisional glance through the codebase there appear to be 3 main use-cases within swarm neo:
Task.getThis()calls to get the current task instance (and yield etc)EpollSelectDispatcher.register(...)calls to register listeners and timersISelectClient, mostly through theISelectListenersubclassLegacy swarm also includes some
unregisterandsetExpiryRegistrationcalls, while the protos (without checking for neo vs. legacy) similarly seem to use just{un}registerandTask.getThis()calls.This all feels high-level enough that it ought to be possible to abstract it away from
EpollSelectDispatcherspecifics, but that begs some questions:is it better to make create an abstraction layer for swarm's specific needs, or to rework ocean to support different options?
if we assume that epoll is the underlying mechanism, can we set things up so that e.g. epoll events can be (un)registered via
EpollSelectDispatcherwhile the eventloop can be run by another library (or vice versa)?how (if at all) can we best integrate with languages other than D?
Thoughts, ideas, and feedback all welcome.