NetMQ socket send and receive with blocking queue, how to get signaled when msg arrived (presumably not using ReceiveReady)?

[dickens-code]

Hi recently I am using NetMQ Router and Dealer socket to work out a high performance message sending and receiving mechanism, as such I tried to put all sending (SendMultipartMessage) and receiving logic in a single loop, run by a single thread.

In order to prevent forever 1ms looping for send and receive, I have implemented a blocking send queue that will receive signals when someone pushes message into that queue

While(IsActive) { While(socket.TryReceiceMultipartMessage(...)) { message processing }

While(blockingQueue.TryDequeue(out msg)) { socket.SendMultipartMessage(msg); } }

I made both send and receive operation thread-blocking if there is nothing to receive/send, so that the thread doesn’t have to loop these logic every 1ms

Consider when send queue is empty and the thread is blocked at TryDequeue() call, and at that moment socket receive msg but the thread can’t serve it.

Ideally I can signal the BlockingQueue to release the thread and serve the received msg, but how can I get an alert from socket that message arrived?

I think I can’t use Poller because it would mean that all send and receive logic have to be performed in Poller events (which I don’t know if it is as efficient as a simple loop)

Is NetMQ socket implemented with some blocking signal property (ManualResetEvent) that I can assign a separate thread to block at it (like WaitOne) and got released when msg arrived?

I am sure there may also efficient way of doing send and receive operations, would you mind sharing that with me as well?

Much appreciated for your attention and advice Dickens

[somdoron]

Use NetMQQueue together with NetMQPoller.

NetMQQueue is single-consumer/multi-producer queue which you can add to a netmq poller.

In your case add both the queue and the sockets to the poller and get signalled when the message is ready.

By the way, if the blocking collection support async/await you can use new NetMQRuntime with receiveAsync

[dickens-code]

Thanks for your prompt reply!

In fact I’m very new to Poller. Would you mind shedding me some light to its performance? How is it compared to simple looping to send/receive msg to/from socket?

Just to make sure I understand it correctly. If I add both socket (NetMQSocket) and queue (NetMQQueue) to poller (NetMQPoller), and when I enqueue sth to queue, OR got msg arrived at socket, both will raise ReceiveReady event?

Sorry keep bothering.

Thanks Dickens

On Fri, 30 Aug 2019 at 8:29 PM, Doron Somech notifications@github.com wrote:

Use NetMQQueue together with NetMQPoller.

NetMQQueue is single-consumer/multi-producer queue which you can add to a netmq poller.

In your case add both the queue and the sockets to the poller and get signalled when the message is ready.

By the way, if the blocking collection support async/await you can use new NetMQRuntime with receiveAsync

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/zeromq/netmq/issues/811?email_source=notifications&email_token=AFSEPE4TJ4QG4DP27QCX37LQHEHBZA5CNFSM4ISMMF62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5RQIGA#issuecomment-526582808, or mute the thread https://github.com/notifications/unsubscribe-auth/AFSEPEYYPLTTJ7Z5ZXGLOD3QHEHBZANCNFSM4ISMMF6Q .

[somdoron]

Yes, you are correct.

The advantage of a poller is that you can loop on multiple sockets and queues.

I suggest to read the docs, as it is all documented.

Also, check out NetMQRuntime, as it is the modern way to do that.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity for 365 days. It will be closed if no further activity occurs within 56 days. Thank you for your contributions.