Confusion around io_uring-udp.c example

[SUPERCILEX]

This line confuses me: https://github.com/axboe/liburing/blob/63342497bec87b72d4b415c8bed930c15ff83b0d/examples/io_uring-udp.c#L255

Isn't that how you indicate EOF for SOCK_SEQPACKET? None of this seems documented anywhere which has been a struggle.

[SUPERCILEX]

I also can't get auxiliary data to show up and I'm not sure what I'm doing wrong. This is the client syscall:

sendmsg(3, {msg_name={sa_family=AF_UNIX, sun_path="/tmp/asaveau.clipboard-history"}, msg_namelen=33, msg_iov=[{iov_base="\0", iov_len=1}], msg_iovlen=1, msg_control=[{cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=SCM_RIGHTS, cmsg_data=[0]}], msg_controllen=24, msg_flags=0}, 0) = 1

But at the other end after parsing out io_uring_recvmsg_out, controllen is zero. Is there anything I'm supposed to do to get it to work? Or an example?

[axboe]

CC @DylanZA who wrote this example.

[DylanZA]

This line confuses me:

https://github.com/axboe/liburing/blob/63342497bec87b72d4b415c8bed930c15ff83b0d/examples/io_uring-udp.c#L255

Isn't that how you indicate EOF for SOCK_SEQPACKET? None of this seems documented anywhere which has been a struggle.

I don't believe so? validate validates that the structure of the single buffer is sane so you don't have buffer overruns

[DylanZA]

@SUPERCILEX did you increase the size of the CONTROLLEN? I suspect that is needed

[SUPERCILEX]

BTW, sorry for not being clear: I'm trying to use the example as a starting point for my own code. I'm not running the example.

---

Ok, I'm thoroughly confused at this point. Let me explain the behavior I'm seeing:

• First, the same buffer seems to be reused which doesn't make any sense. How can the kernel know I'm done with that buffer? Rather I think the flags aren't being set. Not sure what's going on there.
• The result (aka size of the full buffer) is 16 + payload size on the first msg. Once the client closes the connection, the result is 16.
  • Ah! @DylanZA that's why I thought validation was wrong: because the kernel isn't giving me a new buffer, payload len is wrong and then validation rightfully fails because the buffer only includes space for the header.
• Setting controllen doesn't do anything as far as I can tell

let mut hdr = unsafe { mem::zeroed::<libc::msghdr>() };
hdr.msg_controllen = 10;

---

[server/src/reactor.rs:132:21] &msg = RecvMsgOut {
    header: io_uring_recvmsg_out {
        namelen: 0,
        controllen: 0,
        payloadlen: 5,
        flags: 8,
    },
    msghdr_name_len: 0,
    msghdr_control_len: 10,
    name_data: [],
    control_data: [],

[SUPERCILEX]

Control len works!!! Turns out it was silently being ignored because the buffer length was too small. That's deeply annoying if you ask me, why isn't controllen showing the true data size so I can tell that things were truncated?

[SUPERCILEX]

Ok yeah truncation is totally broken in the sense that it doesn't happen at all. If I send 3 file descriptors (which requires 32 bytes), I silently only get 2 of them and io_uring_recvmsg_out says the size is 24. Is that expected?

[SUPERCILEX]

Ok so I ran the example and it does put a new buffer in the flags. Maybe something is broken with SOCK_SEQPACKET?

[SUPERCILEX]

Ok, the truncation stuff was just broken rust: https://github.com/tokio-rs/io-uring/pull/257

And I tweaked the udp example to use SOCK_SEQPACKET and the buffer management worked just fine, so I must have a bug in my code somewhere.

[SUPERCILEX]

Ah yep for anyone wondering bgid and bid are different. :man_facepalming: bid is what's returned in the flags and I was setting it to bgid so obviously the buffers were broken.

[axboe]

Thanks for following up, always good to know what the issue was. Is this something that could be improved in the man pages?

[SUPERCILEX]

Thanks for asking for feedback, that's awesome! I don't think so:

• The docs clearly state that bid is the buffer ID that will be returned in the CQE once consumed
• The controllen truncation was a bug in library code, so not much to be done besides fix it

It was the usual lots of small bugs combining to make a big mess of debugging. :)