03_cat_liburing - Githubissues

Hey there, thanks for putting these examples together. I'm practicing io_uring and you're resources have been invaluable. Your site is currently down though :( I had to use archive.org.

Anyways, I'm having trouble running example 3 on files that are larger than 8530944 bytes. Calculated by sum total of posix_memalign successes before crash which was *8331 BLOCK_SZ** on my machine.

I'm using WSL2 and Ubuntu 20.10 with Linux Kernel 5.4.81-microsoft-standard-WSL2+

My C is really rusty and it's late which is why I'm reaching out.

I ran the build cmd clang -luring -g -fsanitize=address -fno-omit-frame-pointer main.c

followed by ./a.out random_chars_largefile.txt

this was the output

Async readv failed.

=================================================================
==23776==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 33337 byte(s) in 1 object(s) allocated from:
    #0 0x496aad in malloc /home/brian/src/final/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3
    #1 0x4c6d16 in submit_read_request /tmp/cat_libring/main.c:102:28
    #2 0x4c72ea in main /tmp/cat_libring/main.c:156:19
    #3 0x7f804fa5bcb1 in __libc_start_main csu/../csu/libc-start.c:314:16

Indirect leak of 8531968 byte(s) in 2083 object(s) allocated from:
    #0 0x497547 in posix_memalign /home/brian/src/final/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:226:3
    #1 0x4c6de2 in submit_read_request /tmp/cat_libring/main.c:120:13
    #2 0x4c72ea in main /tmp/cat_libring/main.c:156:19
    #3 0x7f804fa5bcb1 in __libc_start_main csu/../csu/libc-start.c:314:16

SUMMARY: AddressSanitizer: 8565305 byte(s) leaked in 2084 allocation(s).

It looks like the offending code is the following

struct file_info *fi = malloc(sizeof(*fi) +
                                            (sizeof(struct iovec) * blocks));

Which causes an indirect leak with

if( posix_memalign(&buf, BLOCK_SZ, BLOCK_SZ)) {

I couldn't really wrap my mind around how to determine what the value for the align parameter should be. I know it had to be a multiple of 2 and larger than sizeof(void *) but how did you derive BLOCK_SZ should be the value? A comment in the code would really help noobs like me out :(

There are a couple of things at play here. First off, since this program is not long running and exits after printing a file, I left out the memory free logic since the operating system should take care of that and closing open files. So, the "leak" is expected.

However, you've hit upon a bug in the program, in that it can't handle files larger than a few MBs. Here's why:

POSIX.1  allows an implementation to place a limit on the number of items that 
can be passed in iov.  An implementation can advertise its limit by defining 
IOV_MAX in <limits.h>  or  at  run  time  via  the  return  value  from 
sysconf(_SC_IOV_MAX).  On modern Linux systems, the limit is 1024.  Back in 
Linux 2.0 days, this limit was 16.

That's from the man 2 readv man page. Since the block size is only 1024 as defined by BLOCK_SZ, large files will use IOV_MAX blocks to cover all file contents and the program fails. You can easily verify this by making BLOCK_SZ larger. You might also have to set the 2nd argument to posix_memalign() to something like 512. This should handle the file of size you've mentioned.

Making this example program able to handle files of all sizes might make it more complicated to understand well and so, I'm inclined to leave this as is. I'm hoping the issue you've raised will serve as an explanation to someone looking for an answer to why it is unable to handle large files.

Thank you for reporting this.

shuveb / io_uring-by-example

03_cat_liburing #10