oxen-io / oxen-mq

Communications layer used for both the Oxen storage server and oxend
https://oxen.io
BSD 3-Clause "New" or "Revised" License
19 stars 35 forks source link

Fix zmq socket limit setting #79

Closed jagerman closed 2 years ago

jagerman commented 2 years ago

MAX_SOCKETS wasn't working properly because ZMQ uses it when the context is initialized, which happens when the first socket is constructed on that context.

For OxenMQ, we had several sockets constructed on the context during OxenMQ construction, which meant the context_t was being initialized during OxenMQ construction, rather than during start(), and so setting MAX_SOCKETS would have no effect and you'd always get the default.

This fixes it by making all the member variable zmq::socket_t's default-constructed, then replacing them with proper zmq::socket_t's during startup() so that we also defer zmq::context_t initialization to the right place.

A second issue found during testing (also fixed here) is that the socket worker threads use to communicate to the proxy could fail if the worker socket creation would violate the zmq max sockets limit, which wound up throwing an uncaught exception and aborting. This pre-initializes (but doesn't connect) all potential worker threads sockets during start() so that the lazily-initialized worker thread will have one already set up rather than having to create a new one (which could fail).