ii64 / gouring

Go io uring syscall implementation without CGO, rewrite on branch v0.4
MIT License
110 stars 13 forks source link

Bad File descriptor #11

Open VEDANTDOKANIA opened 5 months ago

VEDANTDOKANIA commented 5 months ago

go version ;- 1.21

Linux version - 6.5.0-35-generic #35~22.04.1-Ubuntu SMP

When i am making the new ring buffer many time i am getting error of bad file descriptor. Issue occurs randomly there is no fix pattern for the same. Do i need to change some OS related paramters for the same? Or can you suggest the possible problems for the same.

ring, err := gouring.New(32768, 0)

if err != nil {

    panic(err)
}

image

@ii64 Can you please help in the same

ii64 commented 5 months ago

Hi, this can be occured during io_uring_setup because it cannot allocate a memory in the kernel - resulting it returns -1 ENOMEM but it's not caught up on syscall Errno error check and goes to the mmap fd parameter and when that gets executed it returns EBADF.

This is reproducible on C:

#include <stdio.h>
#include <stdlib.h>
#include <sys/syscall.h>
#include <unistd.h>
#include <errno.h>
#include <linux/io_uring.h>

int io_uring_setup(unsigned entries, struct io_uring_params *p)
{
    return (int) syscall(__NR_io_uring_setup, entries, p);
}

typedef struct slot {
    struct io_uring_params p;
    int fd;
} slot_t;

int main()
{
    #define NMEMB 5000
    int slot_cur, fd;
    slot_t *slot;
    slot_t *slots = calloc(NMEMB, sizeof(slot_t));

    for (slot_cur = 0; slot_cur < NMEMB; slot_cur++) {
        slot = &slots[slot_cur];
        slot->p.features = 0;
        fd = io_uring_setup(32768, &slot->p);
        if (fd < 0) {
            printf("at %d\n", slot_cur);
            perror("io_uring_setup");
        }
        slot->fd = fd;
    }

    printf("OK\n");
    return 0;
}

Giving me this result, tho I am not really sure why it can allocate a ring after there's a fail:

15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0}) = -1 ENOMEM (Cannot allocate memory) <0.000747>
at 1859
io_uring_setup: Cannot allocate memory
15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0}) = -1 ENOMEM (Cannot allocate memory) <0.003288>
at 1860
io_uring_setup: Cannot allocate memory
15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0}) = -1 ENOMEM (Cannot allocate memory) <0.004589>
at 1861
io_uring_setup: Cannot allocate memory
15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0, sq_entries=32768, cq_entries=65536, features=IORING_FEAT_SINGLE_MMAP|IORING_FEAT_NODROP|IORING_FEAT_SUBMIT_STABLE|IORING_FEAT_RW_CUR_POS|IORING_FEAT_CUR_PERSONALITY|IORING_FEAT_FAST_POLL|IORING_FEAT_POLL_32BITS|IORING_FEAT_SQPOLL_NONFIXED|IORING_FEAT_EXT_ARG|IORING_FEAT_NATIVE_WORKERS|IORING_FEAT_RSRC_TAGS|IORING_FEAT_CQE_SKIP|IORING_FEAT_LINKED_FILE|IORING_FEAT_REG_REG_RING, sq_off={head=0, tail=4, ring_mask=16, ring_entries=24, flags=36, dropped=32, array=1048640, user_addr=0}, cq_off={head=8, tail=12, ring_mask=20, ring_entries=28, overflow=44, cqes=64, flags=40, user_addr=0}}) = 1575 <0.001254>
15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0, sq_entries=32768, cq_entries=65536, features=IORING_FEAT_SINGLE_MMAP|IORING_FEAT_NODROP|IORING_FEAT_SUBMIT_STABLE|IORING_FEAT_RW_CUR_POS|IORING_FEAT_CUR_PERSONALITY|IORING_FEAT_FAST_POLL|IORING_FEAT_POLL_32BITS|IORING_FEAT_SQPOLL_NONFIXED|IORING_FEAT_EXT_ARG|IORING_FEAT_NATIVE_WORKERS|IORING_FEAT_RSRC_TAGS|IORING_FEAT_CQE_SKIP|IORING_FEAT_LINKED_FILE|IORING_FEAT_REG_REG_RING, sq_off={head=0, tail=4, ring_mask=16, ring_entries=24, flags=36, dropped=32, array=1048640, user_addr=0}, cq_off={head=8, tail=12, ring_mask=20, ring_entries=28, overflow=44, cqes=64, flags=40, user_addr=0}}) = 1576 <0.000680>
15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0}) = -1 ENOMEM (Cannot allocate memory) <0.002103>
at 1864
io_uring_setup: Cannot allocate memory
15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0}) = -1 ENOMEM (Cannot allocate memory) <0.001222>
at 1865
io_uring_setup: Cannot allocate memory
15:57:13 io_uring_setup(32768, {flags=0, sq_thread_cpu=0, sq_thread_idle=0}) = -1 ENOMEM (Cannot allocate memory) <0.000990>
at 1866
io_uring_setup: Cannot allocate memory