Support NodeJS as a platform

[foretspaisibles]

I am maintaining a codebase running lwt on top of NodeJS, thanks to the excellent js_of_ocaml. Of course the Unix parts of lwt are unavailable but I have written replacements for some of the missing functionalities. I am also happy to say that the resulting programs have been put in production for several month now and display a very satisfying stability and ability to handle large amount of data.

I would be very interesting to make NodeJS being supported next to Unix as a platform, as to ease writing platform-independant code. Is there anybody doing the same, or even just vaguely interest by this?

[aantron]

Interested to see what you have made.

Also, it would be nice to run CI tests even of only the existing platform-independent parts of Lwt on Node (see #249). Would be interested to hear about your setup or environment.

[aantron]

I want this. Any word? Ideally, we would make room for several bindings besides Unix, though practically we would implement the ones that people are most interested in first. Ideally, we could make sure Lwt interacts really well with JS, has a nice mapping to promises. We would compile Lwt with Bucklescript and publish Lwt in npm for people using that to depend on. cc @fxfactorial @sgrove @jordwalke ?

[jordwalke]

It's very valuable because many people have existing deployments of node and they can't just drop everything and create an entirely new server environment from scratch. For example, corporate IT departments might have spent a long time vetting infrastructure like node, and therefore that's the easiest way to wedge OCaml into the system.

[mfp]

I've taken a quick look at NodeJS' API and many things in Lwt_unix would be implementable straightforwardly atop it (minus the type equalities, etc., with Unix, which would have to go in the NodeJS-safe version), the easiest ones being the FS functions. Lwt_bytes could be implemented atop Buffer, but it would probably be easier to reimplement higher-level functionality (re: Lwt_io's reorganization). With a good enough coverage of Lwt_unix (needs not be complete and would certainly not include any of the async method stuff or just NOP it) and Lwt_io, things like Ocsigen could be nearly runable on NodeJS.

@michipili are your NodeJS reimplementations of Lwt_unix (and others) functionality available somewhere for reference?

[foretspaisibles]

@mfp @aantron It is not yet. I will prepare a repository for this soon, so that we can start a discussion on concrete basis.

[avsm]

On a similar note, we have a bunch of code in https://github.com/docker/vpnkit/blob/master/src/hostnet/sig.ml that abstracts over Lwt_unix vs libuv. It might be useful to build a 'core' set of Unix signatures that could be implemented by multiple libraries for js_of_ocaml, libuv, etc. (cc @djs55)

[Lupus]

@michipili how's your Lwt-on-nodejs project going after couple years? We do something similar with some abstraction over httpaf for native http stack and node http stack, so that services could be run on top of native OCaml or node platforms.

[foretspaisibles]

That's pretty interesting! On my side, I've not been doing much in the topic these days and do not plan to. I guess I still could have my code somewhere but it would probably not be running without upgrading to modern LWT interface at least!