Passing IO function as FunPtr

[ysangkok]

I am trying to make some libuv bindings, such that I can have asynchronous networking.

But I noted that there is no FunPtr. And even if there were, I am not sure I'd be able to pass Haskell functions as callback to the UV functions.

diff --git a/src/runtime/eval.c b/src/runtime/eval.c
index 6c68c49..fcc0119 100644
--- a/src/runtime/eval.c
+++ b/src/runtime/eval.c
@@ -1115,6 +1115,8 @@ const struct ffi_info ffi_table[] = {
 #if defined(FFI_EXTRA)
 FFI_EXTRA
 #endif  /* defined(FFI_EXTRA) */
+  { "uv_loop_init", (funptr_t) uv_loop_init, FFI_Pi },
+  { "uv_run", (funptr_t) uv_run, FFI_Pii },
 };

 /* Look up an FFI function by name */
@@ -2732,7 +2734,14 @@ execio(NODEPTR *np)
         case FFI_PPP: FFI (2); xp = PTRARG(1);yp = PTRARG(2);  rp = (*(void*   (*)(void*, void*    ))f)(xp,yp); n = mkPtr(rp); RETIO(n);
         case FFI_IPI: FFI (2); xi = INTARG(1);yp = PTRARG(2);  ri = (*(value_t (*)(value_t, void*  ))f)(xi,yp); n = mkInt(ri); RETIO(n);
         case FFI_iPi: FFI (2); xi = INTARG(1);yp = PTRARG(2);  ri = (*(int     (*)(int,   void*    ))f)(xi,yp); n = mkInt(ri); RETIO(n);
+        case FFI_Pii: FFI (2); xp = PTRARG(1);yi = INTARG(2);  ri = (*(int     (*)(void*, int      ))f)(xp,yi); n = mkInt(ri); RETIO(n); // e.g. uv_run
+        case FFI_PiP: FFI (2); xp = PTRARG(1);yi = INTARG(2);  ri = (*(void*   (*)(void*, int      ))f)(xp,yi); n = mkPtr(ri); RETIO(n); // e.g. uv_connection_cb
+        case FFI_PPi: FFI (2); xp = PTRARG(1);yp = PTRARG(2);  ri = (*(int     (*)(void*, void*    ))f)(xp,yp); n = mkInt(ri); RETIO(n); // e.g. uv_tcp_init, uv_accept
         case FFI_iPV: FFI (2); xi = INTARG(1);yp = PTRARG(2);       (*(void    (*)(int,   void*    ))f)(xi,yp);                RETIO(combUnit);
+        case FFI_PiPi:FFI (3); xp = PTRARG(1);yi = INTARG(2); zp = PTRARG(3); (*(int     (*)(void*, int,   void* ))f)(xp,yi,zp); n = mkInt(ri); RETIO(n); // e.g. uv_listen
+        case FFI_PPPi:FFI (3); xp = PTRARG(1);yp = PTRARG(2); zp = PTRARG(3); (*(int     (*)(void*, void*, void* ))f)(xp,yp,zp); n = mkPtr(ri); RETIO(n); // e.g. uv_read_start
+        case FFI_PiPV:FFI (3); xp = PTRARG(1);yi = INTARG(2); zp = PTRARG(3); (*(void    (*)(void*, int,   void* ))f)(xp,yi,zp); RETIO(combUnit); // e.g. uv_alloc_cb, uv_read_cb
+        case FFI_PPii:FFI (3); xp = PTRARG(1);yp = PTRARG(2); zi = INTARG(3); (*(int     (*)(void*, void*, int   ))f)(xp,yp,zi); n = mkInt(ri); RETIO(n); // e.g. uv_tcp_bind
         case FFI_PPzV:FFI (3); xp = PTRARG(1);yp = PTRARG(2); zi = INTARG(3); (*(void    (*)(void*, void*, size_t))f)(xp,yp,zi); RETIO(combUnit);
         case FFI_PIIPI:FFI (4);xp = PTRARG(1);yi = INTARG(2); zi = INTARG(3); wp = PTRARG(4);
           ri = (*(int     (*)(void*, int, int, void*    ))f)(xp,yi,zi,wp); n = mkInt(ri); RETIO(n);

(note the example UV functions in the comments above)

Not sure if I am approaching this wrong. How would you recommend doing asynchronous networking bindings? It is futile to try to do it outside the RTS?

In the above snippet, you can see the signature of uv_connection_cb. I would like to define a Haskell functions that I can pass to e.g. uv_listen: https://docs.libuv.org/en/v1.x/stream.html#c.uv_listen

[ysangkok]

Eric Mertens just informed me that this is part of Haskell 2010, so I suppose it might be out of scope: https://www.haskell.org/onlinereport/haskell2010/haskellch8.html#x15-1610008.5

[augustss]

The MicroHs FFI is a giant hack and lacks many things. I'm going to redo it at some point, but so far I've not had any incentive to do so. Allowing a Haskell function to be passed to C is extra tricky. In the worst case it requires runtime code generation, but not always. I will give it some thought.

[augustss]

UV seems like an interesting library to have bindings to. It could probably be made quite nice using Concurrent Haskell, which I plan to support.

[augustss]

I've redone how FFI works. It's now a bit less fiddle. No need to update a table in eval.c. There is still no FunPtr, but now there is a chance, at least.

[augustss]

Thinking a bit more about callbacks, I don't think that this can easily work. You can't suddenly call a function while MicroHs is executing. For this to be possible, there would have to be multiple threads and execution contexts.

[ysangkok]

Thanks for improving the FFI, I will check it out.

Apropos threading: I regard threading as a way to run blocking functions concurrently. libuv requires setting up callbacks, but the callbacks won't be called until the event loop is run using uv_run, which even takes a parameter to say whether to only run a single poll. So we know when the callback will be called. MicroHs can still have its own entry point (it doesn't need to be all-libuv). Not sure if that actually improves the situation any, I just thought I wanted to point this out, because wasn't sure why you mentioned threading. The UV documentation is slightly confusing since formulation like[0] makes it sound like there is no event loop. But there is. And it can run just one iteration.

But I'd like to note that I think asynchronous I/O can be achieved without FunPtrs or threading, but it will need a bit of C binding code.

Using structs like[1], which can be allocated in MicroHs user code, one can provide a fixed set of C callbacks creators like [2]. It won't be a problem to just linearly search through all read_data structs to see which ones triggered, since it makes little sense to allow more than one read operation queued per stream.

• [0] Callback called when a stream server has received an incoming connection.
• [1]:

  struct read_data {
   bool triggered; /* unless this is set, the callback wasn't triggered yet */
   bool failed; /* when set, this read_data can be deallocated because it won't get filled again (connection died or similar) */
   ssize_t amount_read; /* refers to next field */
   char[8192] read_bytes;
  }

• [2]: uv_connection_cb* mk_uv_connection_cb(struct read_data* dest) and an associated free operation

EDIT: uv_connection_cb above should probably be uv_read_cb

[augustss]

The way you describe the callbacks makes it easier to make it work. It will still need some care for the runtime to handle C calling back into the runtime. I think the easiest way to do this is actually to have Haskell level threads and assign a new thread (actually a HEC as GHC calls it) to the callback.

When I support Concurrent Haskell I will think about dynamically generated FunPtr again.