[V1] Message loss mitigation

[cvellan]

Hi,

While developing a solution using version 1 of iot-edge I noticed some messages that were sent were not being received. I understand that this is a design decision to deal with over-saturated broker queues as described in #381.

Obviously the most reliable solution would be to send less messages and spend less time processing each message, to ensure the backing queues are emptied before they can fill to capacity. But aside from that, is there any setting or pattern that might further mitigate message loss?

Thanks in advance!

[darobs]

Hello @cvellan

I don't have any non-code options for you.

I know when I attempted to play around with performance, the best performance/cost option I found was to increase the size of the send buffer on the publish socket created on line 150 in broker.c:

It didn't guarantee you wouldn't drop packets (which is why we opted not to change the default settings), but the larger you made this buffer, the less likely messages would be dropped. You would set it something like this (after line 150):

const int buffer_size = 256*1024; // (i.e. 256K)
nn_setsockopt (result->publish_socket, NN_PUB, NN_RCVBUF, (const void *)&buffer_size), sizeof(int));

Likewise, you can set the receive buffer on the module sockets in line 363 of broker.c.

These were the best options I had, you may want to look to see if there are any of the default options that might be causing you trouble as well. If you are sending larger messages, you may want to check if NN_RCVMAXSIZE is adequate. PLease check out the nanomsg options here: https://nanomsg.org/v1.1.4/nn_setsockopt.html

[cvellan]

Thanks for that confirmation, and thanks twice for those pointers!