it uses an old build tool (omake) which prevents us from switching everything to dune, removing Cygwin to fix our Windows build and (optionally) vendoring our dependencies with opam monorepo.
Advantages of Luv:
it's likely to become the IO foundation for Lwt in future (Luv and Lwt have the same maintainers)
it builds with dune so will help us remove Cygwin, fix our Windows build and vendor our dependencies
Disadvantages of Luv:
it exposes the raw callback API of libuv so we need to carefully interface with the Lwt event loop
we haven't tested it as thoroughly
vpnkit had already separated out IO into a separate module, which was needed to migrate from Lwt_unix to Uwt in the past. However the Luv interface is a thinner layer on top of libuv which means we need to carefully interface the libuv callback-based world with the Lwt promise world. The proposal here is to run Luv on one thread and Lwt on another, and to use Mutex-protected Queues to send work between the two event loops.
A typical function which can be called from Lwt looks like this:
let do_some_io () : unit Lwt.t =
(* Called from the Lwt universe, which is where the TCP/IP stack runs *)
Luv_lwt.in_luv (fun return ->
(* Now we're in the Luv event loop *)
Luv.Do.something _ (function
(* libuv calls callbacks when interesting things happen.
We want to tell the Lwt caller on the other thread.
The helper function `return` pushes a result on the return queue. *)
| Error err -> return (Error (`Msg (Luv.Error.strerror err)))
| Ok () -> return (Ok ())
)
)
where Luv_lwt.in_luv pushes the function to a queue and signals the Luv event loop, and then when a return result is finally ready, the function return pushes the value to a queue and signals the Lwt event loop. The queues and signalling is all in a small module Luv_lwt.
The callback code is bit nested in places in this style:
match foo () with
| Error _ ->
| Ok () ->
match bar () with
| Error _ ->
see for example binding a UDP socket (you have to expand the collapsed diff of host.ml otherwise it's not visible).
but I thought I'd write everything out before trying to be too clever with combinators.
This was a good opportunity to start writing some inline unit tests to complement the existing end-to-end tests.
It was also a good opportunity to reformat the new code with ocamlformat.
Both Uwt and Luv are bindings to libuv, the scalable IO library used by node.js.
Advantages of
Uwt
:Lwt
-based API for promises so works easily with the rest of our codeDisadvantages of
Uwt
:Advantages of
Luv
:Lwt
in future (Luv
andLwt
have the same maintainers)dune
so will help us remove Cygwin, fix our Windows build and vendor our dependenciesDisadvantages of
Luv
:libuv
so we need to carefully interface with theLwt
event loopvpnkit
had already separated out IO into a separate module, which was needed to migrate fromLwt_unix
toUwt
in the past. However theLuv
interface is a thinner layer on top oflibuv
which means we need to carefully interface thelibuv
callback-based world with theLwt
promise world. The proposal here is to runLuv
on one thread andLwt
on another, and to use Mutex-protected Queues to send work between the two event loops.A typical function which can be called from
Lwt
looks like this:where
Luv_lwt.in_luv
pushes the function to a queue and signals theLuv
event loop, and then when a return result is finally ready, the functionreturn
pushes the value to a queue and signals theLwt
event loop. The queues and signalling is all in a small module Luv_lwt.The callback code is bit nested in places in this style:
see for example binding a UDP socket (you have to expand the collapsed diff of
host.ml
otherwise it's not visible).but I thought I'd write everything out before trying to be too clever with combinators.
This was a good opportunity to start writing some inline unit tests to complement the existing end-to-end tests.
It was also a good opportunity to reformat the new code with
ocamlformat
.Signed-off-by: David Scott dave@recoil.org