axboe / liburing

Library providing helpers for the Linux kernel io_uring support
MIT License
2.84k stars 402 forks source link

Linux 5.13+ Errors #377

Closed YoSTEALTH closed 3 years ago

YoSTEALTH commented 3 years ago

Linux 5.13.1

I was io_uring_prep_write b'hello world' and its writing b'\xf0\x16\xec\xa4\x0e\x7f\x00\x00rld' and b'\x10\x07\xe6\x80\xb5\x7f\x00\x0001\xb7'. So I ran a test and this is whats happened with that.

$ make clean && make -B && make runtests
make[1]: Entering directory '/.../liburing/src'
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Leaving directory '/.../liburing/examples'
Running configure ...
prefix                        /usr
includedir                    /usr/include
libdir                        /usr/lib
libdevdir                     /usr/lib
relativelibdir                
mandir                        /usr/man
datadir                       /usr/share
__kernel_rwf_t                yes
__kernel_timespec             yes
open_how                      no
statx                         yes
C++                           yes
has_ucontext                  yes
CC                            gcc
CXX                           g++
make[1]: Entering directory '/.../liburing/src'
     CC setup.ol
     CC queue.ol
     CC syscall.ol
     CC register.ol
     AR liburing.a
ar: creating liburing.a
 RANLIB liburing.a
     CC setup.os
     CC queue.os
     CC syscall.os
     CC register.os
     CC liburing.so.2.0.0
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
     CC helpers.o
     CC 232c93d07b74-test
     CC 35fa71a030ca-test
     CC 500f9fbadef8-test
     CC 7ad0e4b2f83c-test
     CC 8a9973408177-test
     CC 917257daa0fe-test
     CC a0908ae19763-test
     CC a4c0b3decb33-test
     CC accept
     CC accept-link
     CC accept-reuse
     CC accept-test
     CC across-fork
     CC splice
     CC b19062a56726-test
     CC b5837bd5311d-test
     CC ce593a6c480a-test
     CC close-opath
     CC connect
     CC cq-full
     CC cq-overflow
     CC cq-peek-batch
     CC cq-ready
     CC cq-size
     CC d4ae271dfaae-test
     CC d77a67ed5f27-test
     CC defer
     CC double-poll-crash
double-poll-crash.c: In function ‘main’:
double-poll-crash.c:142:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  142 |   *(uint8_t*)0x20000000 = 6;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:143:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  143 |   *(uint8_t*)0x20000001 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:154:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  154 |   *(uint8_t*)0x2000002c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:155:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  155 |   *(uint8_t*)0x2000002d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:156:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  156 |   *(uint8_t*)0x2000002e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:157:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  157 |   *(uint8_t*)0x2000002f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:158:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  158 |   *(uint8_t*)0x20000030 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:159:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  159 |   *(uint8_t*)0x20000031 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:160:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  160 |   *(uint8_t*)0x20000032 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:161:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  161 |   *(uint8_t*)0x20000033 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:162:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  162 |   *(uint8_t*)0x20000034 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:163:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  163 |   *(uint8_t*)0x20000035 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:164:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  164 |   *(uint8_t*)0x20000036 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:165:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  165 |   *(uint8_t*)0x20000037 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:166:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  166 |   *(uint8_t*)0x20000038 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:167:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  167 |   *(uint8_t*)0x20000039 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:168:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  168 |   *(uint8_t*)0x2000003a = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:169:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  169 |   *(uint8_t*)0x2000003b = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:170:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  170 |   *(uint8_t*)0x2000003c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:171:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  171 |   *(uint8_t*)0x2000003d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:172:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  172 |   *(uint8_t*)0x2000003e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:173:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  173 |   *(uint8_t*)0x2000003f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:180:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  180 |   *(uint8_t*)0x20000090 = 9;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:181:3: warning: ‘memcpy’ offset [0, 18] is out of the bounds [0, 0] [-Warray-bounds]
  181 |   memcpy((void*)0x20000091, "\xaf\x09\x01\xbc\xf9\xc6\xe4\x92\x86\x51\x7d\x7f"
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  182 |                             "\xbd\x43\x7d\x16\x69\x3e\x05",
      |                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  183 |          19);
      |          ~~~
     CC eeed8b54e0df-test
     CC eventfd
     CC eventfd-disable
     CC eventfd-ring
     CC fadvise
     CC fallocate
     CC fc2a85cb02ef-test
     CC file-register
     CC file-update
     CC files-exit-hang-poll
     CC files-exit-hang-timeout
     CC fixed-link
     CC fsync
     CC hardlink
     CC io-cancel
     CC io_uring_enter
     CC io_uring_register
     CC io_uring_setup
     CC iopoll
     CC lfs-openat
     CC lfs-openat-write
     CC link
     CC link-timeout
     CC link_drain
     CC madvise
     CC mkdir
     CC multicqes_drain
     CC nop
     CC nop-all-sizes
     CC open-close
     CC openat2
     CC personality
     CC pipe-eof
     CC pipe-reuse
     CC poll
     CC poll-cancel
     CC poll-cancel-ton
     CC poll-link
     CC poll-many
     CC poll-mshot-update
     CC poll-ring
     CC poll-v-poll
     CC probe
     CC read-write
     CC register-restrictions
     CC rename
     CC ring-leak
     CC ring-leak2
     CC rw_merge_test
     CC self
     CC send_recv
     CC send_recvmsg
     CC shared-wq
     CC short-read
     CC shutdown
     CC sigfd-deadlock
     CC socket-rw
     CC socket-rw-eagain
     CC sq-full
     CC sq-poll-dup
     CC sq-poll-kthread
     CC sq-poll-share
     CC sqpoll-disable-exit
     CC sqpoll-exit-hang
     CC sqpoll-sleep
     CC sq-space_left
     CC stdout
     CC submit-reuse
     CC symlink
     CC teardowns
     CC thread-exit
     CC timeout
     CC timeout-new
     CC timeout-overflow
     CC unlink
     CC wakeup-hang
     CC sendmsg_fs_cve
     CC rsrc_tags
     CC statx
     CXX sq-full-cpp
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
     CC io_uring-test
     CC io_uring-cp
     CC link-cp
     CC ucontext-cp
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/src'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/test'
Running test 232c93d07b74-test
Running test 35fa71a030ca-test
Test 35fa71a030ca-test timed out (may not be a failure)
Running test 500f9fbadef8-test
Running test 7ad0e4b2f83c-test
Running test 8a9973408177-test
Running test 917257daa0fe-test
Running test a0908ae19763-test
Running test a4c0b3decb33-test
Running test accept
Running test accept-link
Running test accept-reuse
Running test accept-test
Running test across-fork
Running test splice
Running test b19062a56726-test
Running test b5837bd5311d-test
Running test ce593a6c480a-test
Running test close-opath
Running test connect
Running test cq-full
Running test cq-overflow
Running test cq-peek-batch
Running test cq-ready
Running test cq-size
Running test d4ae271dfaae-test
Test requires root, skipping
Running test d77a67ed5f27-test
Running test defer
Running test double-poll-crash
Running test eeed8b54e0df-test
open file: Permission denied
Test eeed8b54e0df-test failed with ret 1
Running test eventfd
Running test eventfd-disable
Running test eventfd-ring
Running test fadvise
Running test fallocate
Running test fc2a85cb02ef-test
Test needs failslab/fail_futex/fail_page_alloc enabled, skipped
Running test file-register
Running test file-update
Running test files-exit-hang-poll
Running test files-exit-hang-timeout
Running test fixed-link
Running test fsync
open: Permission denied
test_barrier_fsync failed
Test fsync failed with ret 1
Running test hardlink
linkat not supported, skipping
Running test io-cancel
Running test io_uring_enter
io_uring_enter(3, 1, 0, 4294967295, (nil))
io_uring_enter(-1, 0, 0, 0, (nil))
io_uring_enter(0, 0, 0, 0, (nil))
io_uring_enter(3, 1, 0, 0, (nil))
Allocating 4096 sqes
Submitting 4096 I/Os
Done
Waiting for 4096 events
Reaping 4096 I/Os
Submitting invalid sqe index.
PASS
Running test io_uring_register
RELIMIT_MEMLOCK: 18446744073709551615 (18446744073709551615)
io_uring_register(-1, 0, (nil), 0)
io_uring_register(3, 0, (nil), 0)
io_uring_register(4, 4294967295, (nil), 0)
io_uring_register(4, 0, 0x7fff26220980, 1)
io_uring_register(4, 0, 0x7fff26220980, 1)
io_uring_register(4, 0, 0x7fff26220980, 1)
Unable to map a huge page.  Try increasing /proc/sys/vm/nr_hugepages by at least 1.
Skipping the hugepage test
reserve file-backed buffers
io_uring_register(4, 0, 0x7fff26220980, 1)
io_uring_register(4, 0, 0x7f4567ee4010, 1000000)
io_uring_register(4, 0, 0x7f4567ee4010, 1024)
allocated 17179869180 bytes of address space
io_uring_register(4, IORING_REGISTER_FILES, 0x7f4168e27000, 32767)...succeeded
io_uring_register(4, IORING_UNREGISTER_FILES, 0, 0)...succeeded
io_uring_submit:
    opcode: 6
    flags:  0x00000000
    fd:     4
    poll_events: 0x00000005
io_uring_register(4, 2, 0x7fff262208cc, 1)
PASS
memfd registration isn't supported, skip
Running test io_uring_setup
io_uring_setup(0, 0x7ffc18002fa0), flags: none, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, (nil)), flags: none, feat: none, resv: , sq_thread_cpu: 0
io_uring_setup(1, 0x7ffc18002fa0), flags: none, feat: none, resv: 0x00000001 0x00000001 0x00000001, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffc18002fa0), flags: 0xffffffff, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffc18002fa0), flags: IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffc18002fa0), flags: IORING_SETUP_SQPOLL|IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 16
PASS
Running test iopoll
SQPOLL requires root, skipping
Running test lfs-openat
Running test lfs-openat-write
Running test link
Running test link-timeout
Running test link_drain
open: Permission denied
test_link_drain_one failed
Test link_drain failed with ret 1
Running test madvise
Suspicious timings (53 > 47)
Running test mkdir
mkdirat not supported, skipping
Running test multicqes_drain
Running test nop
Running test nop-all-sizes
Running test open-close
Running test openat2
Running test personality
Not root, skipping
Running test pipe-eof
Running test pipe-reuse
Running test poll
Running test poll-cancel
Running test poll-cancel-ton
Running test poll-link
Running test poll-many
Running test poll-mshot-update
Running test poll-ring
Running test poll-v-poll
Running test probe
Running test read-write
SQPOLL requires root, skipping
Not root, skipping test_write_efbig
Running test register-restrictions
Running test rename
Running test ring-leak
Running test ring-leak2
Running test rw_merge_test
rw_merge_test: rw_merge_test.c:40: main: Assertion `!ret' failed.
timeout: the monitored command dumped core
./runtests.sh: line 66: 111387 Aborted                 timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test rw_merge_test failed with ret 134
Running test self
Running test send_recv
Running test send_recvmsg
Running test shared-wq
Running test short-read
Running test shutdown
Running test sigfd-deadlock
Running test socket-rw
Running test socket-rw-eagain
Running test sq-full
Running test sq-poll-dup
Running test sq-poll-kthread
sqpoll requires root!
test_sq_poll_kthread_stopped_exit: skipped
sqpoll requires root!
test_sq_poll_kthread_stopped_noexit: skipped
Running test sq-poll-share
Running test sqpoll-disable-exit
Running test sqpoll-exit-hang
Running test sqpoll-sleep
Running test sq-space_left
Running test stdout
This is a pipe test
This is a fixed pipe test
Running test submit-reuse
Running test symlink
symlinkat not supported, skipping
Running test teardowns
Running test thread-exit
Running test timeout
Running test timeout-new
Running test timeout-overflow
Skipping
Running test unlink
Running test wakeup-hang
Running test sendmsg_fs_cve
chroot(): Operation not permitted
error: Test failed
Test sendmsg_fs_cve failed with ret 1
Running test rsrc_tags
Running test statx
Running test sq-full-cpp
Tests failed:  <eeed8b54e0df-test> <fsync> <link_drain> <rw_merge_test> <sendmsg_fs_cve>
make[1]: *** [Makefile:294: runtests] Error 1
make[1]: Leaving directory '/.../liburing/test'
make: *** [Makefile:23: runtests] Error 2
YoSTEALTH commented 3 years ago
  1. I install Linux 5.14-rc1 but it wont boot and falls back to 5.13.1
  2. Ran the test again, it went from 5 failed test to 7! Great.
  3. Liburing and io_uring totally needs build bots before release!!!
$ make clean && make -B && make runtests --jobs=8
make[1]: Entering directory '/.../liburing/src'
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Leaving directory '/.../liburing/examples'
Running configure ...
prefix                        /usr
includedir                    /usr/include
libdir                        /usr/lib
libdevdir                     /usr/lib
relativelibdir                
mandir                        /usr/man
datadir                       /usr/share
__kernel_rwf_t                yes
__kernel_timespec             yes
open_how                      no
statx                         yes
C++                           yes
has_ucontext                  yes
CC                            gcc
CXX                           g++
make[1]: Entering directory '/.../liburing/src'
     CC setup.ol
     CC queue.ol
     CC syscall.ol
     CC register.ol
     AR liburing.a
ar: creating liburing.a
 RANLIB liburing.a
     CC setup.os
     CC queue.os
     CC syscall.os
     CC register.os
     CC liburing.so.2.0.0
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
     CC helpers.o
     CC 232c93d07b74-test
     CC 35fa71a030ca-test
     CC 500f9fbadef8-test
     CC 7ad0e4b2f83c-test
     CC 8a9973408177-test
     CC 917257daa0fe-test
     CC a0908ae19763-test
     CC a4c0b3decb33-test
     CC accept
     CC accept-link
     CC accept-reuse
     CC accept-test
     CC across-fork
     CC splice
     CC b19062a56726-test
     CC b5837bd5311d-test
     CC ce593a6c480a-test
     CC close-opath
     CC connect
     CC cq-full
     CC cq-overflow
     CC cq-peek-batch
     CC cq-ready
     CC cq-size
     CC d4ae271dfaae-test
     CC d77a67ed5f27-test
     CC defer
     CC double-poll-crash
double-poll-crash.c: In function ‘main’:
double-poll-crash.c:142:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  142 |   *(uint8_t*)0x20000000 = 6;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:143:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  143 |   *(uint8_t*)0x20000001 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:154:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  154 |   *(uint8_t*)0x2000002c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:155:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  155 |   *(uint8_t*)0x2000002d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:156:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  156 |   *(uint8_t*)0x2000002e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:157:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  157 |   *(uint8_t*)0x2000002f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:158:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  158 |   *(uint8_t*)0x20000030 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:159:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  159 |   *(uint8_t*)0x20000031 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:160:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  160 |   *(uint8_t*)0x20000032 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:161:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  161 |   *(uint8_t*)0x20000033 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:162:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  162 |   *(uint8_t*)0x20000034 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:163:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  163 |   *(uint8_t*)0x20000035 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:164:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  164 |   *(uint8_t*)0x20000036 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:165:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  165 |   *(uint8_t*)0x20000037 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:166:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  166 |   *(uint8_t*)0x20000038 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:167:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  167 |   *(uint8_t*)0x20000039 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:168:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  168 |   *(uint8_t*)0x2000003a = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:169:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  169 |   *(uint8_t*)0x2000003b = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:170:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  170 |   *(uint8_t*)0x2000003c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:171:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  171 |   *(uint8_t*)0x2000003d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:172:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  172 |   *(uint8_t*)0x2000003e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:173:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  173 |   *(uint8_t*)0x2000003f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:180:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  180 |   *(uint8_t*)0x20000090 = 9;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:181:3: warning: ‘memcpy’ offset [0, 18] is out of the bounds [0, 0] [-Warray-bounds]
  181 |   memcpy((void*)0x20000091, "\xaf\x09\x01\xbc\xf9\xc6\xe4\x92\x86\x51\x7d\x7f"
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  182 |                             "\xbd\x43\x7d\x16\x69\x3e\x05",
      |                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  183 |          19);
      |          ~~~
     CC eeed8b54e0df-test
     CC eventfd
     CC eventfd-disable
     CC eventfd-ring
     CC fadvise
     CC fallocate
     CC fc2a85cb02ef-test
     CC file-register
     CC file-update
     CC files-exit-hang-poll
     CC files-exit-hang-timeout
     CC fixed-link
     CC fsync
     CC hardlink
     CC io-cancel
     CC io_uring_enter
     CC io_uring_register
     CC io_uring_setup
     CC iopoll
     CC lfs-openat
     CC lfs-openat-write
     CC link
     CC link-timeout
     CC link_drain
     CC madvise
     CC mkdir
     CC multicqes_drain
     CC nop
     CC nop-all-sizes
     CC open-close
     CC openat2
     CC personality
     CC pipe-eof
     CC pipe-reuse
     CC poll
     CC poll-cancel
     CC poll-cancel-ton
     CC poll-link
     CC poll-many
     CC poll-mshot-update
     CC poll-ring
     CC poll-v-poll
     CC probe
     CC read-write
     CC register-restrictions
     CC rename
     CC ring-leak
     CC ring-leak2
     CC rw_merge_test
     CC self
     CC send_recv
     CC send_recvmsg
     CC shared-wq
     CC short-read
     CC shutdown
     CC sigfd-deadlock
     CC socket-rw
     CC socket-rw-eagain
     CC sq-full
     CC sq-poll-dup
     CC sq-poll-kthread
     CC sq-poll-share
     CC sqpoll-disable-exit
     CC sqpoll-exit-hang
     CC sqpoll-sleep
     CC sq-space_left
     CC stdout
     CC submit-reuse
     CC symlink
     CC teardowns
     CC thread-exit
     CC timeout
     CC timeout-new
     CC timeout-overflow
     CC unlink
     CC wakeup-hang
     CC sendmsg_fs_cve
     CC rsrc_tags
     CC statx
     CXX sq-full-cpp
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
     CC io_uring-test
     CC io_uring-cp
     CC link-cp
     CC ucontext-cp
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/src'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/test'
Running test 232c93d07b74-test
Running test 35fa71a030ca-test
Test 35fa71a030ca-test timed out (may not be a failure)
Running test 500f9fbadef8-test
Running test 7ad0e4b2f83c-test
Running test 8a9973408177-test
Running test 917257daa0fe-test
Running test a0908ae19763-test
Running test a4c0b3decb33-test
Running test accept
Running test accept-link
Running test accept-reuse
Running test accept-test
Running test across-fork
Running test splice
Running test b19062a56726-test
Running test b5837bd5311d-test
Running test ce593a6c480a-test
Running test close-opath
Running test connect
Running test cq-full
Running test cq-overflow
Running test cq-peek-batch
Running test cq-ready
Running test cq-size
Running test d4ae271dfaae-test
Test requires root, skipping
Running test d77a67ed5f27-test
Running test defer
Running test double-poll-crash
Running test eeed8b54e0df-test
open file: Permission denied
Test eeed8b54e0df-test failed with ret 1
Running test eventfd
Running test eventfd-disable
Running test eventfd-ring
Running test fadvise
Running test fallocate
Running test fc2a85cb02ef-test
Test needs failslab/fail_futex/fail_page_alloc enabled, skipped
Running test file-register
Running test file-update
Running test files-exit-hang-poll
Running test files-exit-hang-timeout
Running test fixed-link
Running test fsync
open: Permission denied
test_barrier_fsync failed
Test fsync failed with ret 1
Running test hardlink
linkat not supported, skipping
Running test io-cancel
Running test io_uring_enter
io_uring_enter(5, 1, 0, 4294967295, (nil))
io_uring_enter(-1, 0, 0, 0, (nil))
io_uring_enter(0, 0, 0, 0, (nil))
io_uring_enter(5, 1, 0, 0, (nil))
Allocating 4096 sqes
Submitting 4096 I/Os
Done
Waiting for 4096 events
Reaping 4096 I/Os
Submitting invalid sqe index.
PASS
Running test io_uring_register
RELIMIT_MEMLOCK: 18446744073709551615 (18446744073709551615)
io_uring_register(-1, 0, (nil), 0)
io_uring_register(5, 0, (nil), 0)
io_uring_register(6, 4294967295, (nil), 0)
io_uring_register(6, 0, 0x7ffcf1828740, 1)
io_uring_register(6, 0, 0x7ffcf1828740, 1)
io_uring_register(6, 0, 0x7ffcf1828740, 1)
Unable to map a huge page.  Try increasing /proc/sys/vm/nr_hugepages by at least 1.
Skipping the hugepage test
reserve file-backed buffers
io_uring_register(6, 0, 0x7ffcf1828740, 1)
io_uring_register(6, 0, 0x7f0ba0ff8010, 1000000)
io_uring_register(6, 0, 0x7f0ba0ff8010, 1024)
allocated 17179869180 bytes of address space
io_uring_register(6, IORING_REGISTER_FILES, 0x7f07a1f3b000, 32767)...succeeded
io_uring_register(6, IORING_UNREGISTER_FILES, 0, 0)...succeeded
io_uring_submit:
    opcode: 6
    flags:  0x00000000
    fd:     6
    poll_events: 0x00000005
io_uring_register(6, 2, 0x7ffcf182868c, 1)
PASS
memfd registration isn't supported, skip
Running test io_uring_setup
io_uring_setup(0, 0x7ffd261e3b40), flags: none, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, (nil)), flags: none, feat: none, resv: , sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd261e3b40), flags: none, feat: none, resv: 0x00000001 0x00000001 0x00000001, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd261e3b40), flags: 0xffffffff, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd261e3b40), flags: IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd261e3b40), flags: IORING_SETUP_SQPOLL|IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 16
PASS
Running test iopoll
SQPOLL requires root, skipping
Running test lfs-openat
Running test lfs-openat-write
Running test link
Running test link-timeout
Running test link_drain
open: Permission denied
test_link_drain_one failed
Test link_drain failed with ret 1
Running test madvise
Suspicious timings (53 > 47)
Running test mkdir
mkdirat not supported, skipping
Running test multicqes_drain
Running test nop
Running test nop-all-sizes
Running test open-close
Running test openat2
Running test personality
Not root, skipping
Running test pipe-eof
Running test pipe-reuse
Running test poll
Running test poll-cancel
Running test poll-cancel-ton
Running test poll-link
Running test poll-many
timeout: the monitored command dumped core
./runtests.sh: line 66: 109380 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-many failed with ret 139
Running test poll-mshot-update
timeout: the monitored command dumped core
./runtests.sh: line 66: 109391 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-mshot-update failed with ret 139
Running test poll-ring
Running test poll-v-poll
Running test probe
Running test read-write
SQPOLL requires root, skipping
Not root, skipping test_write_efbig
Running test register-restrictions
Running test rename
Running test ring-leak
Running test ring-leak2
Running test rw_merge_test
rw_merge_test: rw_merge_test.c:40: main: Assertion `!ret' failed.
timeout: the monitored command dumped core
./runtests.sh: line 66: 109542 Aborted                 timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test rw_merge_test failed with ret 134
Running test self
Running test send_recv
Running test send_recvmsg
Running test shared-wq
Running test short-read
Running test shutdown
Running test sigfd-deadlock
Running test socket-rw
Running test socket-rw-eagain
Running test sq-full
Running test sq-poll-dup
Running test sq-poll-kthread
sqpoll requires root!
test_sq_poll_kthread_stopped_exit: skipped
sqpoll requires root!
test_sq_poll_kthread_stopped_noexit: skipped
Running test sq-poll-share
Running test sqpoll-disable-exit
Running test sqpoll-exit-hang
Running test sqpoll-sleep
Running test sq-space_left
Running test stdout
This is a pipe test
This is a fixed pipe test
Running test submit-reuse
Running test symlink
symlinkat not supported, skipping
Running test teardowns
Running test thread-exit
Running test timeout
Running test timeout-new
Running test timeout-overflow
Skipping
Running test unlink
Running test wakeup-hang
Running test sendmsg_fs_cve
chroot(): Operation not permitted
error: Test failed
Test sendmsg_fs_cve failed with ret 1
Running test rsrc_tags
Running test statx
Running test sq-full-cpp
Tests failed:  <eeed8b54e0df-test> <fsync> <link_drain> <poll-many> <poll-mshot-update> <rw_merge_test> <sendmsg_fs_cve>
make[1]: *** [Makefile:294: runtests] Error 1
make[1]: Leaving directory '/.../liburing/test'
make: *** [Makefile:23: runtests] Error 2
YoSTEALTH commented 3 years ago

@axboe @isilence This is the error I am getting while trying to compile using cffi for binding

$ /opt/python/3.10/bin/python3 setup.py build_ext --inplace                          
prefix                        /usr
includedir                    /usr/include
libdir                        /usr/lib
libdevdir                     /usr/lib
relativelibdir                
mandir                        /usr/man
datadir                       /usr/share
__kernel_rwf_t                yes
__kernel_timespec             yes
open_how                      no
statx                         yes
C++                           yes
has_ucontext                  yes
CC                            gcc
CXX                           g++
running build_ext
generating cffi module 'build/temp.linux-x86_64-3.10/liburing._liburing.c'
building 'liburing._liburing' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I./libs/liburing/src/include -I/opt/python/3.10/include/python3.10 -c ./libs/liburing/src/queue.c -o build/temp.linux-x86_64-3.10/./libs/liburing/src/queue.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I./libs/liburing/src/include -I/opt/python/3.10/include/python3.10 -c ./libs/liburing/src/register.c -o build/temp.linux-x86_64-3.10/./libs/liburing/src/register.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I./libs/liburing/src/include -I/opt/python/3.10/include/python3.10 -c ./libs/liburing/src/setup.c -o build/temp.linux-x86_64-3.10/./libs/liburing/src/setup.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I./libs/liburing/src/include -I/opt/python/3.10/include/python3.10 -c ./libs/liburing/src/syscall.c -o build/temp.linux-x86_64-3.10/./libs/liburing/src/syscall.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I./libs/liburing/src/include -I/opt/python/3.10/include/python3.10 -c build/temp.linux-x86_64-3.10/liburing._liburing.c -o build/temp.linux-x86_64-3.10/build/temp.linux-x86_64-3.10/liburing._liburing.o
build/temp.linux-x86_64-3.10/liburing._liburing.c: In function ‘_cffi_const_IORING_RSRC_FILE’:                 
build/temp.linux-x86_64-3.10/liburing._liburing.c:1071:12: error: ‘IORING_RSRC_FILE’ undeclared (first use in this function); did you mean ‘IORING_REGISTER_FILES’?                                                           
 1071 |   int n = (IORING_RSRC_FILE) <= 0;                                                                     
      |            ^~~~~~~~~~~~~~~~                                                                            
      |            IORING_REGISTER_FILES                                                                       
build/temp.linux-x86_64-3.10/liburing._liburing.c:1071:12: note: each undeclared identifier is reported only once for each function it appears in
build/temp.linux-x86_64-3.10/liburing._liburing.c: In function ‘_cffi_const_IORING_RSRC_BUFFER’:               
build/temp.linux-x86_64-3.10/liburing._liburing.c:1078:12: error: ‘IORING_RSRC_BUFFER’ undeclared (first use in this function); did you mean ‘IORING_CQE_F_BUFFER’?                                                           
 1078 |   int n = (IORING_RSRC_BUFFER) <= 0;                                                                   
      |            ^~~~~~~~~~~~~~~~~~                                                                          
      |            IORING_CQE_F_BUFFER                                                                         
build/temp.linux-x86_64-3.10/liburing._liburing.c: In function ‘_cffi_const_IORING_REGISTER_RSRC’:             
build/temp.linux-x86_64-3.10/liburing._liburing.c:1533:12: error: ‘IORING_REGISTER_RSRC’ undeclared (first use in this function); did you mean ‘IORING_REGISTER_LAST’?                                                        
 1533 |   int n = (IORING_REGISTER_RSRC) <= 0;                                                                 
      |            ^~~~~~~~~~~~~~~~~~~~                                                                        
      |            IORING_REGISTER_LAST                                                                        
build/temp.linux-x86_64-3.10/liburing._liburing.c: In function ‘_cffi_const_IORING_REGISTER_RSRC_UPDATE’:      
build/temp.linux-x86_64-3.10/liburing._liburing.c:1540:12: error: ‘IORING_REGISTER_RSRC_UPDATE’ undeclared (first use in this function); did you mean ‘IORING_REGISTER_FILES_UPDATE’?                                         
 1540 |   int n = (IORING_REGISTER_RSRC_UPDATE) <= 0;                                                          
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                 
      |            IORING_REGISTER_FILES_UPDATE                                                                
build/temp.linux-x86_64-3.10/liburing._liburing.c: In function ‘_cffi_checkfld_struct_io_uring_rsrc_register’: 
build/temp.linux-x86_64-3.10/liburing._liburing.c:7167:12: error: ‘struct io_uring_rsrc_register’ has no member named ‘type’                                                                                                  
 7167 |   (void)((p->type) | 0);  /* check that 'struct io_uring_rsrc_register.type' is an integer */          
      |            ^~                                                                                          
build/temp.linux-x86_64-3.10/liburing._liburing.c: In function ‘_cffi_checkfld_struct_io_uring_rsrc_update2’:  
build/temp.linux-x86_64-3.10/liburing._liburing.c:7194:12: error: ‘struct io_uring_rsrc_update2’ has no member named ‘type’                                                                                                   
 7194 |   (void)((p->type) | 0);  /* check that 'struct io_uring_rsrc_update2.type' is an integer */           
      |            ^~                                                                                          
In file included from /usr/include/bits/types/stack_t.h:23,                                                    
                 from /usr/include/signal.h:300,                                                               
                 from ./libs/liburing/src/include/liburing.h:13,                                               
                 from build/temp.linux-x86_64-3.10/liburing._liburing.c:572:                                   
build/temp.linux-x86_64-3.10/liburing._liburing.c: At top level:                                               
build/temp.linux-x86_64-3.10/liburing._liburing.c:7801:13: error: ‘struct io_uring_rsrc_register’ has no member named ‘type’                                                                                                  
 7801 |   { "type", offsetof(struct io_uring_rsrc_register, type),                                             
      |             ^~~~~~~~                                                                                   
build/temp.linux-x86_64-3.10/liburing._liburing.c:7802:56: error: ‘struct io_uring_rsrc_register’ has no member named ‘type’                                                                                                  
 7802 |             sizeof(((struct io_uring_rsrc_register *)0)->type),                                        
      |                                                        ^~                                              
In file included from /usr/include/bits/types/stack_t.h:23,                                                    
                 from /usr/include/signal.h:300,                                                               
                 from ./libs/liburing/src/include/liburing.h:13,                                               
                 from build/temp.linux-x86_64-3.10/liburing._liburing.c:572:                                   
build/temp.linux-x86_64-3.10/liburing._liburing.c:7834:13: error: ‘struct io_uring_rsrc_update2’ has no member named ‘type’                                                                                                   
 7834 |   { "type", offsetof(struct io_uring_rsrc_update2, type),                                              
      |             ^~~~~~~~                                                                                   
build/temp.linux-x86_64-3.10/liburing._liburing.c:7835:55: error: ‘struct io_uring_rsrc_update2’ has no member named ‘type’                                                                                                   
 7835 |             sizeof(((struct io_uring_rsrc_update2 *)0)->type),                                         
      |                                                       ^~                                               
error: command '/usr/bin/gcc' failed with exit code 1
isilence commented 3 years ago

Regarding test failures: Running test fsync\n open: Permission denied. Try to run under root / sudo / with privileges, and then let's see if anything fails. fwiw, it was always a requirement.

isilence commented 3 years ago
build/temp.linux-x86_64-3.10/liburing._liburing.c:1071:12: error: ‘IORING_RSRC_FILE’ undeclared (first use in this function); did you mean ‘IORING_REGISTER_FILES’?                                                           
 1071 |   int n = (IORING_RSRC_FILE) <= 0; 

IORING_RSRC_FILE and others were removed and (have never been released), you won't find it in the sources anywhere. Not sure how cffi works and why would it want them.

YoSTEALTH commented 3 years ago

Try to run under root / sudo / with privileges, and then let's see if anything fails. fwiw, it was always a requirement.

Ran as root, also it shoudln't fail if not in root, it should skip the test.

$ sudo make clean && sudo make -B && sudo make runtests --jobs=8
make[1]: Entering directory '/.../liburing/src'
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Leaving directory '/.../liburing/examples'
Running configure ...
prefix                        /usr
includedir                    /usr/include
libdir                        /usr/lib
libdevdir                     /usr/lib
relativelibdir                
mandir                        /usr/man
datadir                       /usr/share
__kernel_rwf_t                yes
__kernel_timespec             yes
open_how                      no
statx                         yes
C++                           yes
has_ucontext                  yes
CC                            gcc
CXX                           g++
make[1]: Entering directory '/.../liburing/src'
     CC setup.ol
     CC queue.ol
     CC syscall.ol
     CC register.ol
     AR liburing.a
ar: creating liburing.a
 RANLIB liburing.a
     CC setup.os
     CC queue.os
     CC syscall.os
     CC register.os
     CC liburing.so.2.0.0
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
     CC helpers.o
     CC 232c93d07b74-test
     CC 35fa71a030ca-test
     CC 500f9fbadef8-test
     CC 7ad0e4b2f83c-test
     CC 8a9973408177-test
     CC 917257daa0fe-test
     CC a0908ae19763-test
     CC a4c0b3decb33-test
     CC accept
     CC accept-link
     CC accept-reuse
     CC accept-test
     CC across-fork
     CC splice
     CC b19062a56726-test
     CC b5837bd5311d-test
     CC ce593a6c480a-test
     CC close-opath
     CC connect
     CC cq-full
     CC cq-overflow
     CC cq-peek-batch
     CC cq-ready
     CC cq-size
     CC d4ae271dfaae-test
     CC d77a67ed5f27-test
     CC defer
     CC double-poll-crash
double-poll-crash.c: In function ‘main’:
double-poll-crash.c:142:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  142 |   *(uint8_t*)0x20000000 = 6;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:143:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  143 |   *(uint8_t*)0x20000001 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:154:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  154 |   *(uint8_t*)0x2000002c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:155:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  155 |   *(uint8_t*)0x2000002d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:156:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  156 |   *(uint8_t*)0x2000002e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:157:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  157 |   *(uint8_t*)0x2000002f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:158:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  158 |   *(uint8_t*)0x20000030 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:159:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  159 |   *(uint8_t*)0x20000031 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:160:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  160 |   *(uint8_t*)0x20000032 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:161:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  161 |   *(uint8_t*)0x20000033 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:162:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  162 |   *(uint8_t*)0x20000034 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:163:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  163 |   *(uint8_t*)0x20000035 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:164:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  164 |   *(uint8_t*)0x20000036 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:165:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  165 |   *(uint8_t*)0x20000037 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:166:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  166 |   *(uint8_t*)0x20000038 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:167:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  167 |   *(uint8_t*)0x20000039 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:168:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  168 |   *(uint8_t*)0x2000003a = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:169:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  169 |   *(uint8_t*)0x2000003b = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:170:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  170 |   *(uint8_t*)0x2000003c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:171:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  171 |   *(uint8_t*)0x2000003d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:172:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  172 |   *(uint8_t*)0x2000003e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:173:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  173 |   *(uint8_t*)0x2000003f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:180:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  180 |   *(uint8_t*)0x20000090 = 9;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:181:3: warning: ‘memcpy’ offset [0, 18] is out of the bounds [0, 0] [-Warray-bounds]
  181 |   memcpy((void*)0x20000091, "\xaf\x09\x01\xbc\xf9\xc6\xe4\x92\x86\x51\x7d\x7f"
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  182 |                             "\xbd\x43\x7d\x16\x69\x3e\x05",
      |                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  183 |          19);
      |          ~~~
     CC eeed8b54e0df-test
     CC eventfd
     CC eventfd-disable
     CC eventfd-ring
     CC fadvise
     CC fallocate
     CC fc2a85cb02ef-test
     CC file-register
     CC file-update
     CC files-exit-hang-poll
     CC files-exit-hang-timeout
     CC fixed-link
     CC fsync
     CC hardlink
     CC io-cancel
     CC io_uring_enter
     CC io_uring_register
     CC io_uring_setup
     CC iopoll
     CC lfs-openat
     CC lfs-openat-write
     CC link
     CC link-timeout
     CC link_drain
     CC madvise
     CC mkdir
     CC multicqes_drain
     CC nop
     CC nop-all-sizes
     CC open-close
     CC openat2
     CC personality
     CC pipe-eof
     CC pipe-reuse
     CC poll
     CC poll-cancel
     CC poll-cancel-ton
     CC poll-link
     CC poll-many
     CC poll-mshot-update
     CC poll-ring
     CC poll-v-poll
     CC probe
     CC read-write
     CC register-restrictions
     CC rename
     CC ring-leak
     CC ring-leak2
     CC rw_merge_test
     CC self
     CC send_recv
     CC send_recvmsg
     CC shared-wq
     CC short-read
     CC shutdown
     CC sigfd-deadlock
     CC socket-rw
     CC socket-rw-eagain
     CC sq-full
     CC sq-poll-dup
     CC sq-poll-kthread
     CC sq-poll-share
     CC sqpoll-disable-exit
     CC sqpoll-exit-hang
     CC sqpoll-sleep
     CC sq-space_left
     CC stdout
     CC submit-reuse
     CC symlink
     CC teardowns
     CC thread-exit
     CC timeout
     CC timeout-new
     CC timeout-overflow
     CC unlink
     CC wakeup-hang
     CC sendmsg_fs_cve
     CC rsrc_tags
     CC statx
     CXX sq-full-cpp
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
     CC io_uring-test
     CC io_uring-cp
     CC link-cp
     CC ucontext-cp
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/src'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/test'
Running test 232c93d07b74-test:
Running test 35fa71a030ca-test:
Test 35fa71a030ca-test timed out (may not be a failure)
Running test 500f9fbadef8-test:
Running test 7ad0e4b2f83c-test:
Running test 8a9973408177-test:
Running test 917257daa0fe-test:
Running test a0908ae19763-test:
Running test a4c0b3decb33-test:
Running test accept:
Running test accept-link:
Running test accept-reuse:
Running test accept-test:
Running test across-fork:
Running test splice:
Running test b19062a56726-test:
Running test b5837bd5311d-test:
Running test ce593a6c480a-test:
Running test close-opath:
Running test connect:
Running test cq-full:
Running test cq-overflow:
Running test cq-peek-batch:
Running test cq-ready:
Running test cq-size:
Running test d4ae271dfaae-test:
Running test d77a67ed5f27-test:
Running test defer:
Running test double-poll-crash:
Running test eeed8b54e0df-test:
Running test eventfd:
Running test eventfd-disable:
Running test eventfd-ring:
Running test fadvise:
Running test fallocate:
Running test fc2a85cb02ef-test:
Test needs failslab/fail_futex/fail_page_alloc enabled, skipped
Running test file-register:
Running test file-update:
Running test files-exit-hang-poll:
Running test files-exit-hang-timeout:
Running test fixed-link:
Running test fsync:
Running test hardlink:
linkat not supported, skipping
Running test io-cancel:
Running test io_uring_enter:
io_uring_enter(5, 1, 0, 4294967295, (nil))
io_uring_enter(-1, 0, 0, 0, (nil))
io_uring_enter(0, 0, 0, 0, (nil))
io_uring_enter(5, 1, 0, 0, (nil))
Allocating 4096 sqes
Submitting 4096 I/Os
Done
Waiting for 4096 events
Reaping 4096 I/Os
Submitting invalid sqe index.
PASS
Running test io_uring_register:
RELIMIT_MEMLOCK: 18446744073709551615 (18446744073709551615)
io_uring_register(-1, 0, (nil), 0)
io_uring_register(5, 0, (nil), 0)
io_uring_register(6, 4294967295, (nil), 0)
io_uring_register(6, 0, 0x7fff54ea3330, 1)
io_uring_register(6, 0, 0x7fff54ea3330, 1)
io_uring_register(6, 0, 0x7fff54ea3330, 1)
Unable to map a huge page.  Try increasing /proc/sys/vm/nr_hugepages by at least 1.
Skipping the hugepage test
reserve file-backed buffers
io_uring_register(6, 0, 0x7fff54ea3330, 1)
io_uring_register(6, 0, 0x7f72f82fa010, 1000000)
io_uring_register(6, 0, 0x7f72f82fa010, 1024)
allocated 17179869180 bytes of address space
io_uring_register(6, IORING_REGISTER_FILES, 0x7f6ef923d000, 32767)...succeeded
io_uring_register(6, IORING_UNREGISTER_FILES, 0, 0)...succeeded
io_uring_submit:
    opcode: 6
    flags:  0x00000000
    fd:     6
    poll_events: 0x00000005
io_uring_register(6, 2, 0x7fff54ea327c, 1)
PASS
memfd registration isn't supported, skip
Running test io_uring_setup:
io_uring_setup(0, 0x7fffe7a84200), flags: none, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, (nil)), flags: none, feat: none, resv: , sq_thread_cpu: 0
io_uring_setup(1, 0x7fffe7a84200), flags: none, feat: none, resv: 0x00000001 0x00000001 0x00000001, sq_thread_cpu: 0
io_uring_setup(1, 0x7fffe7a84200), flags: 0xffffffff, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7fffe7a84200), flags: IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7fffe7a84200), flags: IORING_SETUP_SQPOLL|IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 16
PASS
Running test iopoll:
Running test lfs-openat:
Running test lfs-openat-write:
Running test link:
Running test link-timeout:
Running test link_drain:
Running test madvise:
Running test mkdir:
mkdirat not supported, skipping
Running test multicqes_drain:
Running test nop:
Running test nop-all-sizes:
Running test open-close:
Running test openat2:
Running test personality:
Running test pipe-eof:
Running test pipe-reuse:
Running test poll:
Running test poll-cancel:
Running test poll-cancel-ton:
Running test poll-link:
Running test poll-many:
timeout: the monitored command dumped core
./runtests.sh: line 66: 218390 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-many failed with ret 139
Running test poll-mshot-update:
timeout: the monitored command dumped core
./runtests.sh: line 66: 218405 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-mshot-update failed with ret 139
Running test poll-ring:
Running test poll-v-poll:
Running test probe:
Running test read-write:
Running test register-restrictions:
Running test rename:
Running test ring-leak:
Running test ring-leak2:
Running test rw_merge_test:
Running test self:
Running test send_recv:
Running test send_recvmsg:
Running test shared-wq:
Running test short-read:
Running test shutdown:
Running test sigfd-deadlock:
Running test socket-rw:
Running test socket-rw-eagain:
Running test sq-full:
Running test sq-poll-dup:
Running test sq-poll-kthread:
Running test sq-poll-share:
Running test sqpoll-disable-exit:
Running test sqpoll-exit-hang:
Running test sqpoll-sleep:
Running test sq-space_left:
Running test stdout:
This is a pipe test
This is a fixed pipe test
Running test submit-reuse:
Running test symlink:
symlinkat not supported, skipping
Running test teardowns:
Running test thread-exit:
Running test timeout:
Running test timeout-new:
Running test timeout-overflow:
Skipping
Running test unlink:
Running test wakeup-hang:
Running test sendmsg_fs_cve:
Running test rsrc_tags:
Running test statx:
Running test sq-full-cpp:
Tests failed:  <poll-many> <poll-mshot-update>
make[1]: *** [Makefile:294: runtests] Error 1
make[1]: Leaving directory '/.../liburing/test'
make: *** [Makefile:23: runtests] Error 2
YoSTEALTH commented 3 years ago

IORING_RSRC_FILE and others were removed and (have never been released), you won't find it in the sources anywhere. Not sure how cffi works and why would it want them.

I see. Double checking. I synced it with this liburing previously, since it was removed it was raising error. I removed it as well and there is no more cffi related error.

isilence commented 3 years ago

Try to run under root / sudo / with privileges, and then let's see if anything fails. fwiw, it was always a requirement.

Ran as root, also it shoudln't fail if not in root, it should skip the test.

The tests are mostly kernel tests, and they served the purpose for us well. I agree that it'd better to skip, as well as there is a lot of space for improvements, just nobody have got there yet. Would be great if someone volunteered...

isilence commented 3 years ago
Running test poll-many:
timeout: the monitored command dumped core
./runtests.sh: line 66: 218390 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-many failed with ret 139
Running test poll-mshot-update:
timeout: the monitored command dumped core
./runtests.sh: line 66: 218405 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-mshot-update failed with ret 139

Interesting, haven't seen it before.

YoSTEALTH commented 3 years ago

@isilence To be honest... Its really bad

def test_bug(tmpdir):
    ring = io_uring()
    cqes = io_uring_cqes(2)
    read = [bytearray(11), bytearray(11), bytearray(11)]
    write = [bytearray(b'hello world'), bytearray(b'hello world'), bytearray(b'hello world')]
    iov_read = iovec(*read)
    iov_wite = iovec(*write)
    i = 0
    try:
        assert io_uring_queue_init(8, ring, 0) == 0

        # open
        sqe = io_uring_get_sqe(ring)
        io_uring_prep_openat(sqe, AT_FDCWD, join(tmpdir, f'file-bug-{i}.txt').encode(), O_CREAT | O_RDWR, 0o700)
        sqe.user_data = 1
        fd = submit_wait_result(ring, cqes)

        # write
        sqe = io_uring_get_sqe(ring)
        io_uring_prep_write(sqe, fd, iov_wite[i].iov_base, iov_wite[i].iov_len, 0)
        sqe.user_data = 2
        assert submit_wait_result(ring, cqes) == 11

        # read
        sqe = io_uring_get_sqe(ring)
        io_uring_prep_read(sqe, fd, iov_read[i].iov_base, iov_read[i].iov_len, 0)
        sqe.user_data = 3
        assert submit_wait_result(ring, cqes) == 11

        print()
        print('read:', read)
        # assert read == write

        # close
        sqe = io_uring_get_sqe(ring)
        io_uring_prep_close(sqe, fd)
        sqe.user_data = 4
        assert submit_wait_result(ring, cqes) == 0

    finally:
        io_uring_queue_exit(ring)

# OUTPUT:
# read: [bytearray(b'@vy\xb8\xf5\x7f\x00\x00rld'),
#        bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
#        bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]

# IT SHOULD OUPUT:
# read: [bytearray(b'hello world'),
#        bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
#        bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]

If a simple write/read can produce wrong results, and its in Linux 5.13.1 stable release. I just hope no one runs this in anything important! It will corrupt their data :-(

Some of my other test, just hang and some just return empty reads.

p.s should mention most of the test does pass without any problems/corruption! its weird.

isilence commented 3 years ago

@isilence To be honest... Its really bad

If a simple write/read can produce wrong results, and its in Linux 5.13.1 stable release. I just hope no one runs this in anything important! It will corrupt their data :-(

Honestly, I wouldn't be so hasty to blame the kernel, as it's a pretty simple case, writev is covered by tests, and there are plenty of python bindings and so places to screw. We have enough of issues reported that are actually userspace bugs.

First, I don't see looping, is there any? It'd explain empty 2nd and 3rd arrays. The corruption looks weird, we'll return to it once we get some free hands, but would recommend to try to debug it. E.g. dumping actual SQEs may help.

YoSTEALTH commented 3 years ago

Honestly, I wouldn't be so hasty to blame the kernel, as it's a pretty simple case, writev is covered by tests, and there are plenty of python bindings and so places to screw. We have enough of issues reported that are actually userspace bugs.

Not blaming, more concerned.

First, I don't see looping, is there any? It'd explain empty 2nd and 3rd arrays.

There was a loop, I removed it to simplify showing whats going on, bah... fixed human error output data :)

Narrowed corruption down to:

def test_bug_1(tmpdir):
    ring = io_uring()
    cqes = io_uring_cqes()
    read = [bytearray(11), bytearray(11), bytearray(11)]
    write = [bytearray(b'hello world'), bytearray(b'hello world'), bytearray(b'hello world')]
    iov_read = iovec(*read)

    # NOTE: works fine
    iov_wite = iovec(*write)
    # OUTPUT:
    # read: [bytearray(b'hello world'), bytearray(b'hello world'), bytearray(b'hello world')]

    # NOTE: corruption
    iov_wite = iovec(bytearray(b'hello world'), bytearray(b'hello world'), bytearray(b'hello world'))
    # OUTPUT:
    # read: [bytearray(b'\xf0\xf7\x8c\x9d\xa5\x7f\x00\x00rld'), bytearray(b'\x00\xf8\x8c\x9d\xa5\x7f\x00\x00rld'), bytearray(b' \xf8\x8c\x9d\xa5\x7f\x00\x00rld')]

    loop = 3
    fds = []
    try:
        assert io_uring_queue_init(8, ring, 0) == 0

        # open
        for i in range(loop):
            sqe = io_uring_get_sqe(ring)
            io_uring_prep_openat(sqe, AT_FDCWD, join(tmpdir, f'file-bug-{i}.txt').encode(), O_CREAT | O_RDWR, 0o700)
            sqe.user_data = i
            fd = submit_wait_result(ring, cqes)
            fds.append(fd)

        # write
        for i in range(loop):
            sqe = io_uring_get_sqe(ring)
            io_uring_prep_write(sqe, fds[i], iov_wite[i].iov_base, iov_wite[i].iov_len, 0)
            sqe.user_data = i
            assert submit_wait_result(ring, cqes) == 11

        # read
        for i in range(loop):
            sqe = io_uring_get_sqe(ring)
            io_uring_prep_read(sqe, fds[i], iov_read[i].iov_base, iov_read[i].iov_len, 0)
            sqe.user_data = i
            assert submit_wait_result(ring, cqes) == 11

        print()
        print('read:', read)
        # confirm
        # for i in range(loop):
        #     assert read[i] == write[i]

        # close
        for i in range(loop):
            sqe = io_uring_get_sqe(ring)
            io_uring_prep_close(sqe, fds[i])
            sqe.user_data = i
            assert submit_wait_result(ring, cqes) == 0
    finally:
        io_uring_queue_exit(ring)

iovec() uses ffi.from_buffer which probably uses memmove or memcpy, most likely the same bug liburing test is raising:

double-poll-crash.c:181:3: warning: ‘memcpy’ offset [0, 18] is out of the bounds [0, 0] [-Warray-bounds]
  181 |   memcpy((void*)0x20000091, "\xaf\x09\x01\xbc\xf9\xc6\xe4\x92\x86\x51\x7d\x7f"
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  182 |                             "\xbd\x43\x7d\x16\x69\x3e\x05",
      |                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  183 |          19);

Can't tell if its the chicken or the egg problem.

isilence commented 3 years ago

It looks like ffi.from_buffer is a magic method returning an address of that object, and it definitely does not copy arrays. Also, seems it doesn't extend lifetime of objects, so the lifetime of unnamed bytearray will end right after iovec(bytearray()) and may get deallocated by python runtime.

isilence commented 3 years ago

Sounds like ffi.from_buffer and iovec are too dangerous to expose as is. If the library isn't considered unsafe (or how it's called in the python world), aka may segfault the program, you'd need to somehow prolong lifetimes. And not only to outlive iovec, but wait until it gets a corresponding CQE.

isilence commented 3 years ago
double-poll-crash.c:181:3: warning: ‘memcpy’ offset [0, 18] is out of the bounds [0, 0] [-Warray-bounds]
  181 |   memcpy((void*)0x20000091, "\xaf\x09\x01\xbc\xf9\xc6\xe4\x92\x86\x51\x7d\x7f"
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  182 |                             "\xbd\x43\x7d\x16\x69\x3e\x05",
      |                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  183 |          19);

It has nothing to do with the issue and actually looks valid, just compiler is confused with syzbot magic.

YoSTEALTH commented 3 years ago

It looks like ffi.from_buffer is a magic method returning an address of that object, and it definitely does not copy arrays. Also, seems it doesn't extend lifetime of objects, so the lifetime of unnamed bytearray will end right after iovec(bytearray()) and may get deallocated by python runtime.

https://cffi.readthedocs.io/en/latest/ref.html#ffi-buffer-ffi-from-buffer

isilence commented 3 years ago

It looks like ffi.from_buffer is a magic method returning an address of that object, and it definitely does not copy arrays. Also, seems it doesn't extend lifetime of objects, so the lifetime of unnamed bytearray will end right after iovec(bytearray()) and may get deallocated by python runtime.

https://cffi.readthedocs.io/en/latest/ref.html#ffi-buffer-ffi-from-buffer

Ok, then it does some referencing. First, it may be that it's confused whether to keep a reference and when to release it, e.g. because iovec is itself a strange ffi object, and the runtime lacks information necessary.

And also, the problem is wider. In the example below the bytearray may get freed awhile is still used the kernel, and so the I/O might unexpectedly fail. Or even worse, the memory may get reused and the kernel will write to an unrelated newly created python object.

def submit():
    b = bytearray();
    iov = iovec(b);
    io_uring_submit(sqe={iov})l

submit();
cqe = wait();
YoSTEALTH commented 3 years ago

I removed 5.13.1 thinking it will boot into 5.12.+ and somehow it booted into 5.14-rc1!!! Ran liburing/test:

$ uname -a
Linux ... 5.14.0-1-MANJARO #1 SMP PREEMPT Mon Jul 12 06:37:07 UTC 2021 x86_64 GNU/Linux

$ sudo make clean && sudo make -B && sudo make runtests --jobs=8
make[1]: Entering directory '/.../liburing/src'
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Leaving directory '/.../liburing/examples'
Running configure ...
prefix                        /usr
includedir                    /usr/include
libdir                        /usr/lib
libdevdir                     /usr/lib
relativelibdir                
mandir                        /usr/man
datadir                       /usr/share
__kernel_rwf_t                yes
__kernel_timespec             yes
open_how                      no
statx                         yes
C++                           yes
has_ucontext                  yes
CC                            gcc
CXX                           g++
make[1]: Entering directory '/.../liburing/src'
     CC setup.ol
     CC queue.ol
     CC syscall.ol
     CC register.ol
     AR liburing.a
ar: creating liburing.a
 RANLIB liburing.a
     CC setup.os
     CC queue.os
     CC syscall.os
     CC register.os
     CC liburing.so.2.0.0
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
     CC helpers.o
     CC 232c93d07b74-test
     CC 35fa71a030ca-test
     CC 500f9fbadef8-test
     CC 7ad0e4b2f83c-test
     CC 8a9973408177-test
     CC 917257daa0fe-test
     CC a0908ae19763-test
     CC a4c0b3decb33-test
     CC accept
     CC accept-link
     CC accept-reuse
     CC accept-test
     CC across-fork
     CC splice
     CC b19062a56726-test
     CC b5837bd5311d-test
     CC ce593a6c480a-test
     CC close-opath
     CC connect
     CC cq-full
     CC cq-overflow
     CC cq-peek-batch
     CC cq-ready
     CC cq-size
     CC d4ae271dfaae-test
     CC d77a67ed5f27-test
     CC defer
     CC double-poll-crash
double-poll-crash.c: In function ‘main’:
double-poll-crash.c:142:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  142 |   *(uint8_t*)0x20000000 = 6;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:143:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  143 |   *(uint8_t*)0x20000001 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:154:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  154 |   *(uint8_t*)0x2000002c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:155:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  155 |   *(uint8_t*)0x2000002d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:156:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  156 |   *(uint8_t*)0x2000002e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:157:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  157 |   *(uint8_t*)0x2000002f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:158:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  158 |   *(uint8_t*)0x20000030 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:159:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  159 |   *(uint8_t*)0x20000031 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:160:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  160 |   *(uint8_t*)0x20000032 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:161:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  161 |   *(uint8_t*)0x20000033 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:162:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  162 |   *(uint8_t*)0x20000034 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:163:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  163 |   *(uint8_t*)0x20000035 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:164:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  164 |   *(uint8_t*)0x20000036 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:165:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  165 |   *(uint8_t*)0x20000037 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:166:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  166 |   *(uint8_t*)0x20000038 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:167:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  167 |   *(uint8_t*)0x20000039 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:168:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  168 |   *(uint8_t*)0x2000003a = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:169:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  169 |   *(uint8_t*)0x2000003b = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:170:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  170 |   *(uint8_t*)0x2000003c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:171:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  171 |   *(uint8_t*)0x2000003d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:172:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  172 |   *(uint8_t*)0x2000003e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:173:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  173 |   *(uint8_t*)0x2000003f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:180:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  180 |   *(uint8_t*)0x20000090 = 9;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:181:3: warning: ‘memcpy’ offset [0, 18] is out of the bounds [0, 0] [-Warray-bounds]
  181 |   memcpy((void*)0x20000091, "\xaf\x09\x01\xbc\xf9\xc6\xe4\x92\x86\x51\x7d\x7f"
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  182 |                             "\xbd\x43\x7d\x16\x69\x3e\x05",
      |                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  183 |          19);
      |          ~~~
     CC eeed8b54e0df-test
     CC eventfd
     CC eventfd-disable
     CC eventfd-ring
     CC fadvise
     CC fallocate
     CC fc2a85cb02ef-test
     CC file-register
     CC file-update
     CC files-exit-hang-poll
     CC files-exit-hang-timeout
     CC fixed-link
     CC fsync
     CC hardlink
     CC io-cancel
     CC io_uring_enter
     CC io_uring_register
     CC io_uring_setup
     CC iopoll
     CC lfs-openat
     CC lfs-openat-write
     CC link
     CC link-timeout
     CC link_drain
     CC madvise
     CC mkdir
     CC multicqes_drain
     CC nop
     CC nop-all-sizes
     CC open-close
     CC openat2
     CC personality
     CC pipe-eof
     CC pipe-reuse
     CC poll
     CC poll-cancel
     CC poll-cancel-ton
     CC poll-link
     CC poll-many
     CC poll-mshot-update
     CC poll-ring
     CC poll-v-poll
     CC probe
     CC read-write
     CC register-restrictions
     CC rename
     CC ring-leak
     CC ring-leak2
     CC rw_merge_test
     CC self
     CC send_recv
     CC send_recvmsg
     CC shared-wq
     CC short-read
     CC shutdown
     CC sigfd-deadlock
     CC socket-rw
     CC socket-rw-eagain
     CC sq-full
     CC sq-poll-dup
     CC sq-poll-kthread
     CC sq-poll-share
     CC sqpoll-disable-exit
     CC sqpoll-exit-hang
     CC sqpoll-sleep
     CC sq-space_left
     CC stdout
     CC submit-reuse
     CC symlink
     CC teardowns
     CC thread-exit
     CC timeout
     CC timeout-new
     CC timeout-overflow
     CC unlink
     CC wakeup-hang
     CC sendmsg_fs_cve
     CC rsrc_tags
     CC statx
     CXX sq-full-cpp
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
     CC io_uring-test
     CC io_uring-cp
     CC link-cp
     CC ucontext-cp
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/src'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/test'
Running test 232c93d07b74-test:
Running test 35fa71a030ca-test:
Test 35fa71a030ca-test timed out (may not be a failure)
Running test 500f9fbadef8-test:
Running test 7ad0e4b2f83c-test:
Running test 8a9973408177-test:
Running test 917257daa0fe-test:
Running test a0908ae19763-test:
Running test a4c0b3decb33-test:
Running test accept:
Running test accept-link:
Running test accept-reuse:
Running test accept-test:
Running test across-fork:
Running test splice:
Running test b19062a56726-test:
Running test b5837bd5311d-test:
Running test ce593a6c480a-test:
Running test close-opath:
Running test connect:
Running test cq-full:
Running test cq-overflow:
Running test cq-peek-batch:
Running test cq-ready:
Running test cq-size:
Running test d4ae271dfaae-test:
Running test d77a67ed5f27-test:
Running test defer:
Running test double-poll-crash:
Running test eeed8b54e0df-test:
Running test eventfd:
Running test eventfd-disable:
Running test eventfd-ring:
Running test fadvise:
Running test fallocate:
Running test fc2a85cb02ef-test:
Test needs failslab/fail_futex/fail_page_alloc enabled, skipped
Running test file-register:
Running test file-update:
Running test files-exit-hang-poll:
Running test files-exit-hang-timeout:
Running test fixed-link:
Running test fsync:
Running test hardlink:
linkat not supported, skipping
Running test io-cancel:
Running test io_uring_enter:
io_uring_enter(5, 1, 0, 4294967295, (nil))
io_uring_enter(-1, 0, 0, 0, (nil))
io_uring_enter(0, 0, 0, 0, (nil))
io_uring_enter(5, 1, 0, 0, (nil))
Allocating 4096 sqes
Submitting 4096 I/Os
Done
Waiting for 4096 events
Reaping 4096 I/Os
Submitting invalid sqe index.
PASS
Running test io_uring_register:
RELIMIT_MEMLOCK: 18446744073709551615 (18446744073709551615)
io_uring_register(-1, 0, (nil), 0)
io_uring_register(5, 0, (nil), 0)
io_uring_register(6, 4294967295, (nil), 0)
io_uring_register(6, 0, 0x7ffef28700a0, 1)
io_uring_register(6, 0, 0x7ffef28700a0, 1)
io_uring_register(6, 0, 0x7ffef28700a0, 1)
Unable to map a huge page.  Try increasing /proc/sys/vm/nr_hugepages by at least 1.
Skipping the hugepage test
reserve file-backed buffers
io_uring_register(6, 0, 0x7ffef28700a0, 1)
io_uring_register(6, 0, 0x7f9c5ebd0010, 1000000)
io_uring_register(6, 0, 0x7f9c5ebd0010, 1024)
allocated 17179869180 bytes of address space
io_uring_register(6, IORING_REGISTER_FILES, 0x7f985fb13000, 32767)...succeeded
io_uring_register(6, IORING_UNREGISTER_FILES, 0, 0)...succeeded
io_uring_submit:
    opcode: 6
    flags:  0x00000000
    fd:     6
    poll_events: 0x00000005
io_uring_register(6, 2, 0x7ffef286ffec, 1)
PASS
Running test io_uring_setup:
io_uring_setup(0, 0x7ffd8a51d000), flags: none, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, (nil)), flags: none, feat: none, resv: , sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd8a51d000), flags: none, feat: none, resv: 0x00000001 0x00000001 0x00000001, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd8a51d000), flags: 0xffffffff, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd8a51d000), flags: IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd8a51d000), flags: IORING_SETUP_SQPOLL|IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 16
PASS
Running test iopoll:
Running test lfs-openat:
Running test lfs-openat-write:
Running test link:
Running test link-timeout:
Running test link_drain:
Running test madvise:
Suspicious timings (66 > 34)
Running test mkdir:
mkdirat not supported, skipping
Running test multicqes_drain:
Running test nop:
Running test nop-all-sizes:
Running test open-close:
Running test openat2:
Running test personality:
Running test pipe-eof:
Running test pipe-reuse:
Running test poll:
Running test poll-cancel:
Running test poll-cancel-ton:
Running test poll-link:
Running test poll-many:
timeout: the monitored command dumped core
./runtests.sh: line 66: 109469 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-many failed with ret 139
Running test poll-mshot-update:
timeout: the monitored command dumped core
./runtests.sh: line 66: 109483 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-mshot-update failed with ret 139
Running test poll-ring:
Running test poll-v-poll:
Running test probe:
Running test read-write:
Running test register-restrictions:
Running test rename:
Running test ring-leak:
Running test ring-leak2:
Running test rw_merge_test:
Running test self:
Running test send_recv:
Running test send_recvmsg:
Running test shared-wq:
Running test short-read:
Running test shutdown:
Running test sigfd-deadlock:
Running test socket-rw:
Running test socket-rw-eagain:
Running test sq-full:
Running test sq-poll-dup:
Running test sq-poll-kthread:
Running test sq-poll-share:
Running test sqpoll-disable-exit:
Running test sqpoll-exit-hang:
Running test sqpoll-sleep:
Running test sq-space_left:
Running test stdout:
This is a pipe test
This is a fixed pipe test
Running test submit-reuse:
Running test symlink:
symlinkat not supported, skipping
Running test teardowns:
Running test thread-exit:
Running test timeout:
Running test timeout-new:
Running test timeout-overflow:
Skipping
Running test unlink:
Running test wakeup-hang:
Running test sendmsg_fs_cve:
Running test rsrc_tags:
Running test statx:
Running test sq-full-cpp:
Tests failed:  <poll-many> <poll-mshot-update>
make[1]: *** [Makefile:294: runtests] Error 1
make[1]: Leaving directory '/.../liburing/test'
make: *** [Makefile:23: runtests] Error 2

I don't think its the cffi issue (at this point), it even says "The original object is kept alive (and, in case of memoryview, locked) as long as the cdata object returned by ffi.from_buffer() is alive." Python doesn't garbage collect till all the reference is gone! Plus I am keeping the origina bytearray around to confirm read/write, so its not going anywhere.

Same corruption exist in 5.14-rc1 as well.

I am giong to try to boot into 5.12 and see what happens.

YoSTEALTH commented 3 years ago
$ uname -a
Linux ... 5.12.16-1-MANJARO #1 SMP PREEMPT Sun Jul 11 13:23:34 UTC 2021 x86_64 GNU/Linux

$ sudo make clean && sudo make -B && sudo make runtests --jobs=8
make[1]: Entering directory '/.../liburing/src'
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Leaving directory '/.../liburing/examples'
Running configure ...
prefix                        /usr
includedir                    /usr/include
libdir                        /usr/lib
libdevdir                     /usr/lib
relativelibdir                
mandir                        /usr/man
datadir                       /usr/share
__kernel_rwf_t                yes
__kernel_timespec             yes
open_how                      no
statx                         yes
C++                           yes
has_ucontext                  yes
CC                            gcc
CXX                           g++
make[1]: Entering directory '/.../liburing/src'
     CC setup.ol
     CC queue.ol
     CC syscall.ol
     CC register.ol
     AR liburing.a
ar: creating liburing.a
 RANLIB liburing.a
     CC setup.os
     CC queue.os
     CC syscall.os
     CC register.os
     CC liburing.so.2.0.0
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
     CC helpers.o
     CC 232c93d07b74-test
     CC 35fa71a030ca-test
     CC 500f9fbadef8-test
     CC 7ad0e4b2f83c-test
     CC 8a9973408177-test
     CC 917257daa0fe-test
     CC a0908ae19763-test
     CC a4c0b3decb33-test
     CC accept
     CC accept-link
     CC accept-reuse
     CC accept-test
     CC across-fork
     CC splice
     CC b19062a56726-test
     CC b5837bd5311d-test
     CC ce593a6c480a-test
     CC close-opath
     CC connect
     CC cq-full
     CC cq-overflow
     CC cq-peek-batch
     CC cq-ready
     CC cq-size
     CC d4ae271dfaae-test
     CC d77a67ed5f27-test
     CC defer
     CC double-poll-crash
double-poll-crash.c: In function ‘main’:
double-poll-crash.c:142:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  142 |   *(uint8_t*)0x20000000 = 6;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:143:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  143 |   *(uint8_t*)0x20000001 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:154:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  154 |   *(uint8_t*)0x2000002c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:155:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  155 |   *(uint8_t*)0x2000002d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:156:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  156 |   *(uint8_t*)0x2000002e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:157:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  157 |   *(uint8_t*)0x2000002f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:158:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  158 |   *(uint8_t*)0x20000030 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:159:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  159 |   *(uint8_t*)0x20000031 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:160:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  160 |   *(uint8_t*)0x20000032 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:161:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  161 |   *(uint8_t*)0x20000033 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:162:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  162 |   *(uint8_t*)0x20000034 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:163:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  163 |   *(uint8_t*)0x20000035 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:164:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  164 |   *(uint8_t*)0x20000036 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:165:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  165 |   *(uint8_t*)0x20000037 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:166:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  166 |   *(uint8_t*)0x20000038 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:167:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  167 |   *(uint8_t*)0x20000039 = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:168:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  168 |   *(uint8_t*)0x2000003a = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:169:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  169 |   *(uint8_t*)0x2000003b = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:170:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  170 |   *(uint8_t*)0x2000003c = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:171:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  171 |   *(uint8_t*)0x2000003d = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:172:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  172 |   *(uint8_t*)0x2000003e = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:173:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  173 |   *(uint8_t*)0x2000003f = 0;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:180:25: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  180 |   *(uint8_t*)0x20000090 = 9;
      |   ~~~~~~~~~~~~~~~~~~~~~~^~~
double-poll-crash.c:181:3: warning: ‘memcpy’ offset [0, 18] is out of the bounds [0, 0] [-Warray-bounds]
  181 |   memcpy((void*)0x20000091, "\xaf\x09\x01\xbc\xf9\xc6\xe4\x92\x86\x51\x7d\x7f"
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  182 |                             "\xbd\x43\x7d\x16\x69\x3e\x05",
      |                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  183 |          19);
      |          ~~~
     CC eeed8b54e0df-test
     CC eventfd
     CC eventfd-disable
     CC eventfd-ring
     CC fadvise
     CC fallocate
     CC fc2a85cb02ef-test
     CC file-register
     CC file-update
     CC files-exit-hang-poll
     CC files-exit-hang-timeout
     CC fixed-link
     CC fsync
     CC hardlink
     CC io-cancel
     CC io_uring_enter
     CC io_uring_register
     CC io_uring_setup
     CC iopoll
     CC lfs-openat
     CC lfs-openat-write
     CC link
     CC link-timeout
     CC link_drain
     CC madvise
     CC mkdir
     CC multicqes_drain
     CC nop
     CC nop-all-sizes
     CC open-close
     CC openat2
     CC personality
     CC pipe-eof
     CC pipe-reuse
     CC poll
     CC poll-cancel
     CC poll-cancel-ton
     CC poll-link
     CC poll-many
     CC poll-mshot-update
     CC poll-ring
     CC poll-v-poll
     CC probe
     CC read-write
     CC register-restrictions
     CC rename
     CC ring-leak
     CC ring-leak2
     CC rw_merge_test
     CC self
     CC send_recv
     CC send_recvmsg
     CC shared-wq
     CC short-read
     CC shutdown
     CC sigfd-deadlock
     CC socket-rw
     CC socket-rw-eagain
     CC sq-full
     CC sq-poll-dup
     CC sq-poll-kthread
     CC sq-poll-share
     CC sqpoll-disable-exit
     CC sqpoll-exit-hang
     CC sqpoll-sleep
     CC sq-space_left
     CC stdout
     CC submit-reuse
     CC symlink
     CC teardowns
     CC thread-exit
     CC timeout
     CC timeout-new
     CC timeout-overflow
     CC unlink
     CC wakeup-hang
     CC sendmsg_fs_cve
     CC rsrc_tags
     CC statx
     CXX sq-full-cpp
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
     CC io_uring-test
     CC io_uring-cp
     CC link-cp
     CC ucontext-cp
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/src'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/src'
make[1]: Entering directory '/.../liburing/test'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/test'
make[1]: Entering directory '/.../liburing/examples'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/.../liburing/examples'
make[1]: Entering directory '/.../liburing/test'
Running test 232c93d07b74-test:
Running test 35fa71a030ca-test:
Test 35fa71a030ca-test timed out (may not be a failure)
Running test 500f9fbadef8-test:
Running test 7ad0e4b2f83c-test:
Running test 8a9973408177-test:
Running test 917257daa0fe-test:
Running test a0908ae19763-test:
Running test a4c0b3decb33-test:
Running test accept:
Running test accept-link:
Test accept-link failed dmesg check
Running test accept-reuse:
Running test accept-test:
Running test across-fork:
Running test splice:
Running test b19062a56726-test:
Running test b5837bd5311d-test:
Running test ce593a6c480a-test:
Running test close-opath:
Running test connect:
Running test cq-full:
Running test cq-overflow:
Running test cq-peek-batch:
Running test cq-ready:
Running test cq-size:
Running test d4ae271dfaae-test:
Running test d77a67ed5f27-test:
Running test defer:
Running test double-poll-crash:
Running test eeed8b54e0df-test:
Running test eventfd:
Running test eventfd-disable:
Running test eventfd-ring:
Running test fadvise:
Running test fallocate:
Running test fc2a85cb02ef-test:
Test needs failslab/fail_futex/fail_page_alloc enabled, skipped
Running test file-register:
Running test file-update:
Running test files-exit-hang-poll:
Running test files-exit-hang-timeout:
Running test fixed-link:
Running test fsync:
Running test hardlink:
linkat not supported, skipping
Running test io-cancel:
Running test io_uring_enter:
io_uring_enter(5, 1, 0, 4294967295, (nil))
io_uring_enter(-1, 0, 0, 0, (nil))
io_uring_enter(0, 0, 0, 0, (nil))
io_uring_enter(5, 1, 0, 0, (nil))
Allocating 4096 sqes
Submitting 4096 I/Os
Done
Waiting for 4096 events
Reaping 4096 I/Os
Submitting invalid sqe index.
PASS
Running test io_uring_register:
RELIMIT_MEMLOCK: 18446744073709551615 (18446744073709551615)
io_uring_register(-1, 0, (nil), 0)
io_uring_register(5, 0, (nil), 0)
io_uring_register(6, 4294967295, (nil), 0)
io_uring_register(6, 0, 0x7ffd2e415fa0, 1)
io_uring_register(6, 0, 0x7ffd2e415fa0, 1)
io_uring_register(6, 0, 0x7ffd2e415fa0, 1)
Unable to map a huge page.  Try increasing /proc/sys/vm/nr_hugepages by at least 1.
Skipping the hugepage test
reserve file-backed buffers
io_uring_register(6, 0, 0x7ffd2e415fa0, 1)
io_uring_register(6, 0, 0x7f1011327010, 1000000)
io_uring_register(6, 0, 0x7f1011327010, 1024)
allocated 17179869180 bytes of address space
io_uring_register(6, IORING_REGISTER_FILES, 0x7f0c1226a000, 32767)...succeeded
io_uring_register(6, IORING_UNREGISTER_FILES, 0, 0)...succeeded
io_uring_submit:
    opcode: 6
    flags:  0x00000000
    fd:     6
    poll_events: 0x00000005
io_uring_register(6, 2, 0x7ffd2e415eec, 1)
PASS
memfd registration isn't supported, skip
Running test io_uring_setup:
io_uring_setup(0, 0x7ffd09a56b40), flags: none, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, (nil)), flags: none, feat: none, resv: , sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd09a56b40), flags: none, feat: none, resv: 0x00000001 0x00000001 0x00000001, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd09a56b40), flags: 0xffffffff, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd09a56b40), flags: IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 0
io_uring_setup(1, 0x7ffd09a56b40), flags: IORING_SETUP_SQPOLL|IORING_SETUP_SQ_AFF, feat: none, resv: 0x00000000 0x00000000 0x00000000, sq_thread_cpu: 16
PASS
Running test iopoll:
Running test lfs-openat:
Running test lfs-openat-write:
Running test link:
Running test link-timeout:
Running test link_drain:
Running test madvise:
Suspicious timings (58 > 42)
Running test mkdir:
mkdirat not supported, skipping
Running test multicqes_drain:
Submitted only 1
test_simple_drain failed
Submitted only 2
test_generic_drain failed
Test multicqes_drain failed with ret 1
Running test nop:
Running test nop-all-sizes:
Running test open-close:
Running test openat2:
Running test personality:
Running test pipe-eof:
Running test pipe-reuse:
Running test poll:
Running test poll-cancel:
Running test poll-cancel-ton:
Running test poll-link:
Running test poll-many:
timeout: the monitored command dumped core
./runtests.sh: line 66: 27069 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-many failed with ret 139
Running test poll-mshot-update:
timeout: the monitored command dumped core
./runtests.sh: line 66: 27082 Segmentation fault      timeout -s INT -k $TIMEOUT $TIMEOUT ./$test_name $dev
Test poll-mshot-update failed with ret 139
Running test poll-ring:
Running test poll-v-poll:
Running test probe:
Running test read-write:
Running test register-restrictions:
Running test rename:
Running test ring-leak:
Running test ring-leak2:
Running test rw_merge_test:
Running test self:
Running test send_recv:
Running test send_recvmsg:
Running test shared-wq:
Running test short-read:
Running test shutdown:
Running test sigfd-deadlock:
Running test socket-rw:
Running test socket-rw-eagain:
Running test sq-full:
Running test sq-poll-dup:
Running test sq-poll-kthread:
Running test sq-poll-share:
Running test sqpoll-disable-exit:
Running test sqpoll-exit-hang:
Running test sqpoll-sleep:
Running test sq-space_left:
Running test stdout:
This is a pipe test
This is a fixed pipe test
Running test submit-reuse:
Running test symlink:
symlinkat not supported, skipping
Running test teardowns:
Running test thread-exit:
Running test timeout:
Running test timeout-new:
Running test timeout-overflow:
Skipping
Running test unlink:
Running test wakeup-hang:
Running test sendmsg_fs_cve:
Running test rsrc_tags:
doesn't support rsrc tags, skip
Running test statx:
Running test sq-full-cpp:
Tests failed:  <accept-link> <multicqes_drain> <poll-many> <poll-mshot-update>
make[1]: *** [Makefile:294: runtests] Error 1
make[1]: Leaving directory '/.../liburing/test'
make: *** [Makefile:23: runtests] Error 2

Ok, then it does some referencing. First, it may be that it's confused whether to keep a reference and when to release it, e.g. because iovec is itself a strange ffi object, and the runtime lacks information necessary.

And also, the problem is wider. In the example below the bytearray may get freed awhile is still used the kernel, and so the I/O might unexpectedly fail. Or even worse, the memory may get reused and the kernel will write to an unrelated newly created python object.

def submit():
    b = bytearray();
    iov = iovec(b);
    io_uring_submit(sqe={iov})l

submit();
cqe = wait();

@isilence you are totally right about this point. b will go away once it exists out of function. Or if you use iovec(bytearray(...)) directly and no reference holding on to bytearray. It can be quite a gotcha. I was trying out silly things to narrow down the cause of the bug. I will double check all my test cases for this reference issues. Thank you.

isilence commented 3 years ago

@YoSTEALTH, you're welcome!

Running test accept-link:
Test accept-link failed dmesg check

Can you paste what dmesg shows? Can be that some patches haven't got backported to 5.12.

YoSTEALTH commented 3 years ago

$ uname -a
Linux ... 5.12.16-1-MANJARO #1 SMP PREEMPT Sun Jul 11 13:23:34 UTC 2021 x86_64 GNU/Linux

$ sudo dmesg --clear
$ sudo make clean && sudo make -B && sudo make runtests --jobs=8
...
Tests failed:  <accept-link> <multicqes_drain> <poll-many> <poll-mshot-update>
...

$ sudo dmesg

[ 3534.338853] audit: type=1106 audit(1626351776.855:122): pid=5456 uid=1000 auid=1000 ses=3 msg='op=PAM:session_close grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3534.338957] audit: type=1104 audit(1626351776.855:123): pid=5456 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.790317] audit: type=1101 audit(1626351803.308:124): pid=5458 uid=1000 auid=1000 ses=3 msg='op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct="..." exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.790802] audit: type=1110 audit(1626351803.308:125): pid=5458 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.794629] audit: type=1105 audit(1626351803.312:126): pid=5458 uid=1000 auid=1000 ses=3 msg='op=PAM:session_open grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.966775] audit: type=1106 audit(1626351803.485:127): pid=5458 uid=1000 auid=1000 ses=3 msg='op=PAM:session_close grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.966971] audit: type=1104 audit(1626351803.485:128): pid=5458 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.982145] audit: type=1101 audit(1626351803.498:129): pid=5469 uid=1000 auid=1000 ses=3 msg='op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct="..." exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.982491] audit: type=1110 audit(1626351803.498:130): pid=5469 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3560.985833] audit: type=1105 audit(1626351803.502:131): pid=5469 uid=1000 auid=1000 ses=3 msg='op=PAM:session_open grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3593.246816] audit: type=1106 audit(1626351835.765:132): pid=5469 uid=1000 auid=1000 ses=3 msg='op=PAM:session_close grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3593.246945] audit: type=1104 audit(1626351835.765:133): pid=5469 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3593.262195] audit: type=1101 audit(1626351835.778:134): pid=6121 uid=1000 auid=1000 ses=3 msg='op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct="..." exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3593.262585] audit: type=1110 audit(1626351835.778:135): pid=6121 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3593.266010] audit: type=1105 audit(1626351835.781:136): pid=6121 uid=1000 auid=1000 ses=3 msg='op=PAM:session_open grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3593.373823] Running test 232c93d07b74-test:
[ 3597.819530] Running test 35fa71a030ca-test:
[ 3657.827843] Running test 500f9fbadef8-test:
[ 3659.352840] Running test 7ad0e4b2f83c-test:
[ 3660.376073] Running test 8a9973408177-test:
[ 3660.497249] Running test 917257daa0fe-test:
[ 3660.519750] Running test a0908ae19763-test:
[ 3660.564585] Running test a4c0b3decb33-test:
[ 3666.785572] Running test accept:
[ 3668.041865] Running test accept-link:
[ 3668.249779] ------------[ cut here ]------------
[ 3668.249785] refcount_t: underflow; use-after-free.
[ 3668.249798] WARNING: CPU: 0 PID: 0 at lib/refcount.c:28 refcount_warn_saturate+0xa6/0xf0
[ 3668.249809] Modules linked in: rfkill mousedev joydev ext4 crc16 mbcache jbd2 iTCO_wdt intel_pmc_bxt intel_powerclamp iTCO_vendor_support coretemp gpio_ich kvm_intel snd_hda_codec_realtek snd_hda_codec_generic kvm ledtrig_audio snd_hda_codec_hdmi irqbypass snd_hda_intel intel_cstate snd_intel_dspcfg snd_intel_sdw_acpi intel_uncore snd_hda_codec pcspkr e1000e snd_hda_core snd_hwdep snd_pcm snd_timer snd i2c_i801 ioatdma soundcore dca i2c_smbus lpc_ich acpi_cpufreq mac_hid i7core_edac uinput nvidia_drm(POE) nvidia_modeset(POE) drm_kms_helper cec syscopyarea sysfillrect sysimgblt fb_sys_fops nvidia(POE) drm agpgart vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) fuse crypto_user bpf_preload ip_tables x_tables hid_logitech_hidpp hid_logitech_dj btrfs blake2b_generic libcrc32c crc32c_generic xor raid6_pq usbhid ata_generic pata_acpi ata_piix arcmsr crc32c_intel
[ 3668.249897] CPU: 0 PID: 0 Comm: swapper/0 Tainted: P           OE     5.12.16-1-MANJARO #1
[ 3668.249901] Hardware name: Supermicro X8DAL/X8DAL, BIOS 2.1        12/30/2011
[ 3668.249903] RIP: 0010:refcount_warn_saturate+0xa6/0xf0
[ 3668.249909] Code: 05 1a 96 6f 01 01 e8 09 2c 59 00 0f 0b c3 80 3d 08 96 6f 01 00 75 95 48 c7 c7 f0 4b 9b ab c6 05 f8 95 6f 01 01 e8 ea 2b 59 00 <0f> 0b c3 80 3d e7 95 6f 01 00 0f 85 72 ff ff ff 48 c7 c7 48 4c 9b
[ 3668.249912] RSP: 0018:ffffb7a180003e18 EFLAGS: 00010086
[ 3668.249916] RAX: 0000000000000000 RBX: ffff936ec146f500 RCX: 0000000000000027
[ 3668.249918] RDX: ffff936e39a186e8 RSI: 0000000000000001 RDI: ffff936e39a186e0
[ 3668.249921] RBP: ffff936ec146e500 R08: 0000000000000000 R09: ffffb7a180003c48
[ 3668.249923] R10: ffffb7a180003c40 R11: ffff936fbffb0b28 R12: ffff936df810fcc0
[ 3668.249925] R13: ffff936df810f800 R14: ffff936ec146e55c R15: 0000000000000001
[ 3668.249928] FS:  0000000000000000(0000) GS:ffff936e39a00000(0000) knlGS:0000000000000000
[ 3668.249931] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3668.249933] CR2: 00005572e1c5fb80 CR3: 00000000afa10000 CR4: 00000000000006f0
[ 3668.249937] Call Trace:
[ 3668.249940]  <IRQ>
[ 3668.249941]  io_link_timeout_fn+0x31f/0x3c0
[ 3668.249951]  ? io_req_defer_prep.part.0+0x230/0x230
[ 3668.249956]  __hrtimer_run_queues+0x12a/0x2c0
[ 3668.249963]  hrtimer_interrupt+0x110/0x2c0
[ 3668.249968]  timer_interrupt+0x14/0x20
[ 3668.249973]  __handle_irq_event_percpu+0x40/0x190
[ 3668.249979]  handle_irq_event+0x58/0xb0
[ 3668.249984]  handle_edge_irq+0x96/0x260
[ 3668.249989]  __common_interrupt+0x41/0xa0
[ 3668.249993]  common_interrupt+0x7e/0xa0
[ 3668.249998]  </IRQ>
[ 3668.250000]  asm_common_interrupt+0x1e/0x40
[ 3668.250007] RIP: 0010:cpuidle_enter_state+0xc4/0x380
[ 3668.250014] Code: 8b ff 65 8b 3d 25 d7 1f 55 e8 08 b4 8b ff 49 89 c5 66 66 66 66 90 31 ff e8 29 c1 8b ff 45 84 ff 0f 85 da 01 00 00 fb 66 66 90 <66> 66 90 45 85 f6 0f 88 11 01 00 00 49 63 d6 4c 2b 2c 24 48 8d 04
[ 3668.250017] RSP: 0018:ffffffffac003e50 EFLAGS: 00000282
[ 3668.250020] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000018
[ 3668.250022] RDX: 00004d7944ffd1bc RSI: 000000000010b90e RDI: ffffffffac007a40
[ 3668.250025] RBP: ffff936e39a36d08 R08: 0000035614b93ec7 R09: 0000035614d0fe00
[ 3668.250027] R10: 0000000000000a9f R11: 0000000000000831 R12: ffffffffac144d60
[ 3668.250029] R13: 0000035614b93ec7 R14: 0000000000000004 R15: 0000000000000001
[ 3668.250037]  cpuidle_enter+0x29/0x40
[ 3668.250041]  do_idle+0x1da/0x270
[ 3668.250046]  cpu_startup_entry+0x19/0x20
[ 3668.250050]  start_kernel+0x871/0x896
[ 3668.250057]  secondary_startup_64_no_verify+0xc2/0xcb
[ 3668.250066] ---[ end trace f4f1e1775a881cbd ]---
[ 3668.266271] Running test accept-reuse:
[ 3668.288896] Running test accept-test:
[ 3668.315580] Running test across-fork:
[ 3668.346451] Running test splice:
[ 3668.420036] Running test b19062a56726-test:
[ 3668.440155] Running test b5837bd5311d-test:
[ 3668.473521] Running test ce593a6c480a-test:
[ 3669.552995] Running test close-opath:
[ 3669.574759] Running test connect:
[ 3669.581522] TCP: request_sock_TCP: Possible SYN flooding on port 62086. Sending cookies.  Check SNMP counters.
[ 3669.596422] Running test cq-full:
[ 3669.616944] Running test cq-overflow:
[ 3669.647280] Running test cq-peek-batch:
[ 3669.666616] Running test cq-ready:
[ 3669.689639] Running test cq-size:
[ 3669.712259] Running test d4ae271dfaae-test:
[ 3669.804422] Running test d77a67ed5f27-test:
[ 3669.975560] Running test defer:
[ 3673.000600] Running test double-poll-crash:
[ 3673.024101] Running test eeed8b54e0df-test:
[ 3673.047152] Running test eventfd:
[ 3673.083204] Running test eventfd-disable:
[ 3673.125797] Running test eventfd-ring:
[ 3673.214810] Running test fadvise:
[ 3673.252946] Running test fallocate:
[ 3673.274396] Running test fc2a85cb02ef-test:
[ 3673.294275] Running test file-register:
[ 3677.833082] Running test file-update:
[ 3678.111939] Running test files-exit-hang-poll:
[ 3679.133883] Running test files-exit-hang-timeout:
[ 3680.155348] Running test fixed-link:
[ 3680.205989] Running test fsync:
[ 3680.231516] Running test hardlink:
[ 3680.250392] Running test io-cancel:
[ 3682.391513] Running test io_uring_enter:
[ 3682.430073] Running test io_uring_register:
[ 3683.772747] Running test io_uring_setup:
[ 3683.796605] Running test iopoll:
[ 3684.908964] Running test lfs-openat:
[ 3684.930340] Running test lfs-openat-write:
[ 3684.952023] Running test link:
[ 3685.005933] Running test link-timeout:
[ 3686.534403] Running test link_drain:
[ 3686.884435] Running test madvise:
[ 3687.009297] Running test mkdir:
[ 3687.031120] Running test multicqes_drain:
[ 3687.086292] Running test nop:
[ 3687.105545] Running test nop-all-sizes:
[ 3687.138717] Running test open-close:
[ 3687.159336] Running test openat2:
[ 3687.184230] Running test personality:
[ 3687.207242] Running test pipe-eof:
[ 3687.228805] Running test pipe-reuse:
[ 3687.252044] Running test poll:
[ 3687.269325] Running test poll-cancel:
[ 3687.289117] Running test poll-cancel-ton:
[ 3687.355143] Running test poll-link:
[ 3687.579424] Running test poll-many:
[ 3687.600278] poll-many[30230]: segfault at 40000 ip 00005594379f7886 sp 00007ffda1e6bb58 error 4 in poll-many[5594379f6000+2000]
[ 3687.600295] Code: 00 00 00 00 48 83 ec 08 41 89 f3 e8 b4 fd ff ff 44 89 da 48 83 c4 08 89 c6 e9 c6 fa ff ff 66 0f 1f 44 00 00 48 8b 07 45 31 c0 <8b> 08 8b 47 44 8d 50 01 89 d6 29 ce 48 8b 4f 18 3b 31 77 14 48 8b
[ 3687.600422] audit: type=1701 audit(1626351930.114:137): auid=1000 uid=0 gid=0 ses=3 pid=30230 comm="poll-many" exe="/.../liburing/test/poll-many" sig=11 res=1
[ 3687.889632] audit: type=1334 audit(1626351930.404:138): prog-id=33 op=LOAD
[ 3687.890277] audit: type=1334 audit(1626351930.404:139): prog-id=34 op=LOAD
[ 3687.890718] audit: type=1334 audit(1626351930.407:140): prog-id=35 op=LOAD
[ 3687.892182] audit: type=1130 audit(1626351930.407:141): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@0-30231-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 3687.961765] audit: type=1131 audit(1626351930.477:142): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@0-30231-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[ 3687.992430] audit: type=1701 audit(1626351930.507:143): auid=1000 uid=0 gid=0 ses=3 pid=30229 comm="timeout" exe="/usr/bin/timeout" sig=11 res=1
[ 3687.994652] Running test poll-mshot-update:
[ 3688.026249] poll-mshot-upda[30247]: segfault at 40000 ip 00005625e82a9a36 sp 00007ffc7b71f468 error 4 in poll-mshot-update[5625e82a8000+2000]
[ 3688.026269] Code: 00 00 00 00 48 83 ec 08 41 89 f3 e8 b4 fd ff ff 44 89 da 48 83 c4 08 89 c6 e9 c6 fa ff ff 66 0f 1f 44 00 00 48 8b 07 45 31 c0 <8b> 08 8b 47 44 8d 50 01 89 d6 29 ce 48 8b 4f 18 3b 31 77 14 48 8b
[ 3688.026316] audit: type=1701 audit(1626351930.541:144): auid=1000 uid=0 gid=0 ses=3 pid=30247 comm="poll-mshot-upda" exe="/.../liburing/test/poll-mshot-update" sig=11 res=1
[ 3688.373343] Running test poll-ring:
[ 3688.395267] Running test poll-v-poll:
[ 3692.778717] Running test probe:
[ 3692.801719] Running test read-write:
[ 3697.392928] Running test register-restrictions:
[ 3698.177890] Running test rename:
[ 3698.201435] Running test ring-leak:
[ 3698.364606] Running test ring-leak2:
[ 3699.388688] Running test rw_merge_test:
[ 3699.408808] Running test self:
[ 3699.432207] Running test send_recv:
[ 3699.514088] Running test send_recvmsg:
[ 3699.537406] Running test shared-wq:
[ 3699.560154] Running test short-read:
[ 3699.582036] Running test shutdown:
[ 3699.602264] Running test sigfd-deadlock:
[ 3699.623792] Running test socket-rw:
[ 3699.644783] Running test socket-rw-eagain:
[ 3699.665876] Running test sq-full:
[ 3699.687123] Running test sq-poll-dup:
[ 3701.562088] Running test sq-poll-kthread:
[ 3703.788772] Running test sq-poll-share:
[ 3705.787886] Running test sqpoll-disable-exit:
[ 3706.044275] Running test sqpoll-exit-hang:
[ 3707.067677] Running test sqpoll-sleep:
[ 3707.089107] Running test sq-space_left:
[ 3707.108320] Running test stdout:
[ 3707.299459] Running test submit-reuse:
[ 3708.888040] Running test symlink:
[ 3708.906557] Running test teardowns:
[ 3708.932762] Running test thread-exit:
[ 3708.952645] Running test timeout:
[ 3715.009208] Running test timeout-new:
[ 3717.444217] Running test timeout-overflow:
[ 3717.469019] Running test unlink:
[ 3717.493731] Running test wakeup-hang:
[ 3719.520077] Running test sendmsg_fs_cve:
[ 3719.546241] Running test rsrc_tags:
[ 3719.599848] Running test statx:
[ 3719.624197] Running test sq-full-cpp:
[ 3719.649659] kauditd_printk_skb: 12 callbacks suppressed
[ 3719.649664] audit: type=1106 audit(1626351962.164:157): pid=6121 uid=1000 auid=1000 ses=3 msg='op=PAM:session_close grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3719.649810] audit: type=1104 audit(1626351962.164:158): pid=6121 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3774.026526] audit: type=1101 audit(1626352016.540:159): pid=32392 uid=1000 auid=1000 ses=3 msg='op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct="..." exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3774.026841] audit: type=1110 audit(1626352016.540:160): pid=32392 uid=1000 auid=1000 ses=3 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_env,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[ 3774.030225] audit: type=1105 audit(1626352016.543:161): pid=32392 uid=1000 auid=1000 ses=3 msg='op=PAM:session_open grantors=pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
isilence commented 3 years ago

For the accept-link problem, sent out, something will get into a stable kernel release at some moment.

YoSTEALTH commented 3 years ago

For the accept-link problem, sent out, something will get into a stable kernel release at some moment.

Cool. Liburing is suppose to be usable since Linux 5.1, it would be nice if GitHub Actions https://github.com/axboe/liburing/actions was setup for this account and just have it run runtests

isilence commented 3 years ago

For the accept-link problem, sent out, something will get into a stable kernel release at some moment.

Cool. Liburing is suppose to be usable since Linux 5.1, it would be nice if GitHub Actions https://github.com/axboe/liburing/actions was setup for this account and just have it run runtests

liburing tests are a bit special in a sense that most of them are kernel tests, so we can't use usual automatisation like userspace programs use. E.g. because there is no way to recover after a failure but to reboot, and for other reasons.