RandomAccess.Write throws System.IO.IOException: Invalid argument when buffers count is greater than 1024

Description

Even though this exception sounds very similar to https://github.com/dotnet/runtime/issues/108322#issue-2552756015 , it is actually a very different issue. It occurs in the Unix implementation of the RandomAccess.WriteGatherAtOffset method This is the stack trace

System.IO.IOException: Invalid argument
at System.IO.RandomAccess.WriteGatherAtOffset(SafeFileHandle handle, IReadOnlyList`1 buffers, Int64 fileOffset)

Looking at the implementation I have noticed that we call Interop.Sys.PWriteV passing the buffersCount as a parameter

https://github.com/dotnet/runtime/blob/01aa3d96bb2160144f167b1065e081521d133b48/src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs#L199-L202

which in turn calls the native implementation https://github.com/dotnet/runtime/blob/c049b850a75ccf654fb762d89e0132fd999e4cc1/src/native/libs/System.Native/pal_io.c#L2000

According to the documentation of pwritev here http://man.he.net/man2/pwritev we cannot pass buffersCount greater than 1024

NOTES
       POSIX.1  allows  an  implementation  to  place a limit on the number of
       items that can be passed in iov.  An implementation can  advertise  its
       limit  by  defining IOV_MAX in <limits.h> or at run time via the return
       value from sysconf(_SC_IOV_MAX).  On modern Linux systems, the limit is
       1024.  Back in Linux 2.0 days, this limit was 16.

Reproduction Steps

Call RandomAccess.Write(handle, buffers, fileOffset) by passing list of buffers where buffers are more than 1024

Expected behavior

It should save the buffers to file

Actual behavior

It throws System.IO.IOException: Invalid argument

Regression?

Not tested on .NET6

Known Workarounds

You can split the buffers into smaller lists (<=1024 elements) and call RandomAccess.Write multiple times with the proper offset

Configuration

Which version of .NET is the code running on? .NET7 What OS and version, and what distro if applicable? Ubuntu 22.04.3 LTS under WSL2 What is the architecture (x64, x86, ARM, ARM64)? x64 Do you know whether it is specific to that configuration? It is specific to the Unix implementation of the RandomAccess.WriteGatherAtOffset method

Other information

Haven't tested it, but this should also affect Read(SafeFileHandle handle, IReadOnlyList<Memory<byte>> buffers, long fileOffset) which calls ReadScatterAtOffset(SafeFileHandle handle, IReadOnlyList<Memory<byte>> buffers, long fileOffset)

Tagging subscribers to this area: @dotnet/area-system-io See info in area-owners.md if you want to be subscribed.

Hi @miroslavp according to the documentation that you've linked we can see the following statements just below the "note" section:

Historical C library/kernel differences To deal with the fact that IOV_MAX was so low on early versions of Linux, the glibc wrapper functions for readv() and writev() did some extra work if they detected that the underlying kernel system call failed because this limit was exceeded. In the case of readv(), the wrapper function allocated a temporary buffer large enough for all of the items specified by iov, passed that buffer in a call to read(2), copied data from the buffer to the locations specified by the iov_base fields of the elements of iov, and then freed the buffer. The wrapper function for writev() performed the analogous task using a temporary buffer and a call to write(2).
   The need for this extra effort in the glibc wrapper functions went away
   with Linux 2.2 and later.  However, glibc continued to provide this be-
   havior until version 2.10.  Starting with glibc version 2.9, the  wrap-
   per  functions  provide  this behavior only if the library detects that
   the system is running a Linux kernel older than version 2.6.18 (an  ar-
   bitrarily  selected  kernel  version).  And since glibc 2.20 (which re-
   quires a minimum Linux kernel version of  2.6.32),  the  glibc  wrapper
   functions always just directly invoke the system calls.

Given the context cited above there are two possible solutions in my opinion:

Reimplement the wrapper code in the native part of pal_io.c just like glibc was used to work with version < 2.10
Properly handle the error where buffers are larger than IOV_MAX and fail correctly

I'd like to have an opinion from @adamsitnik that was really helpful in the previous issue.

Thanks guys for your time !

dotnet / runtime