IORING_SETUP_SQPOLL high CPU

[pyhd]

In my case, io_uring is used to replace UDP send() , all work fine without SQPOLL:

sqe = io_uring_get_sqe(ring);
io_uring_prep_send(sqe, p->fd, data, size, msg_flags);
ret = io_uring_submit_and_wait(ring, 1);
if (ret <= 0)
    return -1;
ret = io_uring_peek_cqe(ring, &cqe);
if (ret < 0)
    return -1;
io_uring_cqe_seen(ring, cqe);

But CPU would blow up if I setup it with IORING_SETUP_SQPOLL:

struct io_uring_params params;
memset(&params, 0, sizeof(params));
params.flags = IORING_SETUP_SQPOLL | IORING_FEAT_SQPOLL_NONFIXED;
params.sq_thread_idle = 2000;
ret = io_uring_queue_init_params(2, ring, &params);

Platform: Ubuntu 22.04.2, kernel 6.1.22-060122-generic, liburing 2.3-3

In the strace log, io_uring_enter indeed collected events and periodically tried to wakeup SQ, but the work queue was not seen in ps, I guess the work queue was not created, right? Even so, the program still worked, probably kernel just falled back to the regular syscall.

[isilence]

What was the question? SQPOLL indeed does SQ polling and can take one entire CPU / core polling (but also executing requests). You set sq_thread_idle to 2s, so the SQ thread probably never sleeps.

[isilence]

As for io_uring_queue_init*(), there is no automatic fallback there. If it fails it fails, returns an error and you won't be able to execute anything with it. Did you have some fallback mechanism in your code?

[pyhd]

I have tried to tune sq_thread_idle to 10ms, so the cpu would calm down somehow easily. But in the ps list, wq cannot be found, no matter what sq_thread_idle is. And there is no fallback in the code, so I am sure io_uring is working, even it's double-checked in the strace log.

Is it expected that SQPOLL would take up 100% of a core to poll requests? I thought it would trigger the wq to handle once a new request arrived, probably like epoll ?

My idea is to batch several UDP send() in a single syscall.

[isilence]

Let me add a usual disclaimer, SQPOLL is not a silver bullet nor "make it better" flag, majority of users would most probably be better off not using it at all, it wastes CPU, may be not cache friendly, and definitely needs tuning.

I have tried to tune sq_thread_idle to 10ms, so the cpu would calm down somehow easily. But in the ps list, wq cannot be found, no matter what sq_thread_idle is. And there is no fallback in the code, so I am sure io_uring is working, even it's double-checked in the strace log.

Is it expected that SQPOLL would take up 100% of a core to poll requests? I thought it would trigger the wq to handle once a new request arrived, probably like epoll ?

Well, io-wq would also consume the same amount of CPU + overhead, but no, the SQPOLL thread will try to serve most requests.

My idea is to batch several UDP send() in a single syscall.

FWIW, as noted, if SQPOLL is not a requirement of yours for some reason, it might be better to turn it off and maybe experiment with it in the future once your app works well with normal rings.