JustinTulloss / zeromq.node

Node.js bindings to the zeromq library
MIT License
1.65k stars 284 forks source link

Memory leak on blocking send sockets #247

Open utvara opened 11 years ago

utvara commented 11 years ago

Pardon my coffee but here is a basic example:

zmq = require 'zmq'

sender = zmq.socket 'push', {hwm: 10}
sender.connect "tcp://127.0.0.1:3001"

send = () ->
    sender.send 'message'
    console.log(sender._outgoing.length);

send() for i in [1..100000]

setInterval ->
    console.log process.memoryUsage()
, 1000

It is dangerous to use blocking sockets with hard water mark. In example if there is no pull server connected what will happen is that 10 messages will be queued by zmq and rest will pile up in sender._outgoing. I'm not sure if this is desired but the effect is that upstream goes into slow death mode (memory leak) without downstream server.

Also I've noticed that once pull server is connected socket._outgoing is drained really slow.

This affects also req and dealer type sockets.

Since we are not in the blocking business maybe it would be better to emit an error if zmq socket is blocked due to hwm than to have memory leak.

At least I believe we need to be aware of this behaviour.

utvara commented 11 years ago

Dirty solution for PUSH:

For push example you can detect if hwm is reached by checking sender._ioevents & zmq.ZMQ_POLLOUT before sending the message and handle the problem in app rather than waiting for memory to leak out.