Koromix / koffi

Fast and easy-to-use C FFI module for Node.js
https://koffi.dev/
MIT License
169 stars 3 forks source link

[koffi] Too many callbacks are in use (max = 1024) #18

Closed wanglin86769 closed 1 year ago

wanglin86769 commented 1 year ago

EPICS PV (Process Variable) values can be subscribed to monitor instead of poll, and for each PV, two callbacks are needed for monitoring, they are connection state change callback and value change callback, which means only 512 PVs can be monitored at the same time with koffi. However, there are usually more than 100000 PVs in the EPICS control system of a particle accelerator facility.

Why does koffi restrict callback count? Is it possible to increase the count or make it configurable?

EPICS home page is as follows, https://epics.anl.gov/

Koromix commented 1 year ago

To implement C callbacks in dynamic FFI code, you need to use trampolines, which are small functions that receive the call and redirect it to Koffi.

The only thing Koffi (and other FFI libraries) can use to differentiate two callbacks is the address of the call itself. You don't control the C code and so you can't use any kind of argument; only the raw address of the callback function.

Given these limitations, there are two ways to provide callbacks in dynamic FFI libraries:

The former option is more complex and can fail on some platforms that prevent remapping writeable memory to executable memory (such as Linux PaX builds, OpenBSD, etc.). So Koffi uses the latter approach and has a set of hard-coded 1024 trampolines for FFI callbacks.

So you can't configure it, but I can increase this limit. The main downside is bigger binaries.

Koromix commented 1 year ago

I've increased the limit to 8192 in Koffi 2.3.21-beta.2. Install with npm install koffi@beta.

wanglin86769 commented 1 year ago

Wonderful, thanks.