Closed P3GLEG closed 3 years ago
There are a couple of things at play here. First off, since this program is not long running and exits after printing a file, I left out the memory free logic since the operating system should take care of that and closing open files. So, the "leak" is expected.
However, you've hit upon a bug in the program, in that it can't handle files larger than a few MBs. Here's why:
POSIX.1 allows an implementation to place a limit on the number of items that
can be passed in iov. An implementation can advertise its limit by defining
IOV_MAX in <limits.h> or at run time via the return value from
sysconf(_SC_IOV_MAX). On modern Linux systems, the limit is 1024. Back in
Linux 2.0 days, this limit was 16.
That's from the man 2 readv
man page. Since the block size is only 1024 as defined by BLOCK_SZ, large files will use IOV_MAX blocks to cover all file contents and the program fails. You can easily verify this by making BLOCK_SZ larger. You might also have to set the 2nd argument to posix_memalign()
to something like 512
. This should handle the file of size you've mentioned.
Making this example program able to handle files of all sizes might make it more complicated to understand well and so, I'm inclined to leave this as is. I'm hoping the issue you've raised will serve as an explanation to someone looking for an answer to why it is unable to handle large files.
Thank you for reporting this.
Hey there, thanks for putting these examples together. I'm practicing io_uring and you're resources have been invaluable. Your site is currently down though :( I had to use archive.org.
Anyways, I'm having trouble running example 3 on files that are larger than 8530944 bytes. Calculated by sum total of
posix_memalign
successes before crash which was *8331 BLOCK_SZ** on my machine.I'm using
WSL2
andUbuntu 20.10
with Linux Kernel5.4.81-microsoft-standard-WSL2+
My C is really rusty and it's late which is why I'm reaching out.
I ran the build cmd
clang -luring -g -fsanitize=address -fno-omit-frame-pointer main.c
followed by
./a.out random_chars_largefile.txt
this was the output
It looks like the offending code is the following
Which causes an indirect leak with
I couldn't really wrap my mind around how to determine what the value for the
align
parameter should be. I know it had to be a multiple of 2 and larger thansizeof(void *)
but how did you deriveBLOCK_SZ
should be the value? A comment in the code would really help noobs like me out :(