Open edwintorok opened 2 years ago
Yes this is mentioned in point 6. of the design notes.
E.g.
select
should probably be provided by the scheduler implementation instead of being hardcoded inFunix
.
The fiber scheduler should not be concerned about this though. The idea is that it doesn't know that file descriptors exist.
(which AFAICT is a thin wrapper around the underlying OS event notification mechanisms without also reimplementing other APIs like
luv
does)
That is indeed the motivating factor behind poll
. It wraps kqueue on bsd/macOS (i'll note that the kqueue pieces have only been tested on macOS so it might have some issues on freebsd/openbsd etc), epoll on linux and wepoll
(its an epoll facade on top of iocp so should scale better than Unix.select) on windows. From what I can see in affect
's api, (and something I'm really happy to see, as now I don't have to implement a high level scheduler to test some ideas with poll) it should be possible even in the current version to use something like poll
using a block
function that internally registers file descriptors to poll and uses it to wait for an event?
From what I can see in
affect
's api, (and something I'm really happy to see, as now I don't have to implement a high level scheduler to test some ideas with poll) it should be possible even in the current version to use something likepoll
using ablock
function that internally registers file descriptors to poll and uses it to wait for an event?
Absolutely ! As mentioned in the docs Funix
is an example on how to interface affect
's Fiber
with an event loop (select
for now), it shouldn't be hard to see how you can go about using your library with affect's Fiber
in the same way.
I will still try to play in making Funix
more event loop independent. But by all means try to interface your poll library directly the way Funix
did with Unix
's select
(you'll likely need to do much less since you can register timers directly). It could also be useful to measure the overhead of making Funix
more abstract.
And get back to me if you get in trouble with the proposed interface.
And get back to me if you get in trouble with the proposed interface
I will definitely report back with my experience using affect
with poll
. I found out recently that the multicore backend works on arm64
now so I can install it and try affect
. It'll take me some time to test things with poll
as I'll have to find another way for conditional compilation temporarily as ppx_optcomp
isn't yet supported on OCaml 5.0.0.
Currently
Funix
uses Unix.select: https://github.com/dbuenzli/affect/blob/5ccf1e2fea197f6257ccebffaff8ca0970a0d86c/src/funix.ml#L172 This will limit the application to using at most 1024 file descriptors (limitation in underlyingselect
), which can be quite problematic in practice (aside from performance issues when watching lots of file descriptors). (Note that the limitation isn't in 1024 watched file descriptors at once in Unix.select, but the file descriptor number, so if file descriptors are opened anywhere else that causes one of the FDs used by select to go above 1024 thenselect
will fail.)poll
/epoll
overcomes the limitation in the number of file descriptors: Lwt provides a way to useepoll
throughlibev
,Luv
provides a way to use it vialibuv
, andeio
via io_uring.It would be useful to provide a way to override this and avoid any use of
Unix.select
in an application. E.g.select
should probably be provided by the scheduler implementation instead of being hardcoded inFunix
. An API based on registering/deregistering file descriptors would also be more efficient than working with a list of them.affect
should probably not tie itself to any particular OS / OS wrapper library, but it'd be nice if it could be used together with something like https://anuragsoni.github.io/poll/poll/Poll/index.html (which AFAICT is a thin wrapper around the underlying OS event notification mechanisms without also reimplementing other APIs likeluv
does), and https://ocaml-multicore.github.io/ocaml-uring/uring/Uring/index.html