eclipse-iceoryx / iceoryx

Eclipse iceoryx™ - true zero-copy inter-process-communication
https://iceoryx.io
Apache License 2.0
1.57k stars 373 forks source link

Iceoryx RAW_SOCKET buffer usage #2268

Closed jmyvalour closed 2 months ago

jmyvalour commented 2 months ago

Operating system: Ubuntu 22.04 LTS

Compiler version: clang 18

Eclipse iceoryx version: b18411364d1cac72422c53a4b254fad0c9061204

Hello,

We started monitoring our socket buffer usage and got a lot of warning coming from most of our component using iceoryx, I discovered that iceoryx is using sockets to communicate with the main roudi node and this is these one triggering the warnings, These warning are generated when we it sock_rcvqueue_full kernel tracepoint.

lsof is giving me the following information: PID USER FD TYPE DEVICE SIZE/OFF NODE NAME 509266 root 4u unix 0xffff9eb9d28a5500 0t0 5004732 type=DGRAM ->INO=4202092 437730,net,7u 509266 root 6u unix 0xffff9eb9d28a1100 0t0 5004733 /tmp/process type=DGRAM ->INO=5130274 437730,net,12u

These are the socket information I could retrieve from the one iceoryx is using, family: AF_INET protocol: IPPROTO_UDP type: SOCK_RAW

That is really surprising to us and make no sense to me as messages are going through SHM anyway, is there anything we are missing that could explain this high socket usage ? Any hint, input would help us a lot here to understand what's going on underneath, could not find much information regarding socket from the documentation, I may be missing something really basic here but I can't understand what is happening so far.

Thank you for your help, Jeremy

elBoberido commented 2 months ago

Sockets are used by the runtime to register itself at RouDi and to request endpoints like publisher. Up until November last year it was also used to periodically send the heartbeat for alive detection but this should not be the case anymore in the commit you provided.

Can you check if you get the same results with current master?

jmyvalour commented 2 months ago

Thank you for the provided answer, that make sense now regarding the socket, thank you for the explanation. We will update to current master and see if we keep having these trace point triggered,

elBoberido commented 2 months ago

Well, in theory your commit should already cointain the changes for the heartbeat. That's why I'm wondering why you observe this heavy socket usage. There is this little hope that you use an older commit but I fear you found an interesting problem

elBoberido commented 2 months ago

@jmyvalour from a comment in #2193 I assume the issue with the sockets disappeared after updating to current main. Can you confirm?

jmyvalour commented 2 months ago

Hello,

I confirm the issue is fixed with updated iceoryx version, we were using a previous release version in our environement, apologies for the mix up here, Thank you for your kind support,

jmyvalour commented 2 months ago

closing

elBoberido commented 2 months ago

@jmyvalour no problem. Happy the problem is solved 😃