[Feature] zero-copy ArrayPoolBufferWriter<byte>.WriteTo(Stream)

[vask-msft]

Overview

ArrayPoolBufferWriter<byte>: copy-free write out to a stream

Current behavior

Before the change, the only way to write out the underlying byte array to a stream requires a buffer copy.

Proposed change

Add a new method to write the array to a given stream.

See also https://github.com/vask-msft/CommunityToolkit-dotnet/tree/vask/WriteToStream --- I'll be happy to work it into an accepted PR if the feature is approved.

API breakdown

namespace CommunityToolkit.HighPerformance.Buffers;

public sealed class ArrayPoolBufferWriter<T> : IBuffer<T>, IMemoryOwner<T>
{
...
    /// <summary>
    /// Zero-copy output of the underlying memory to the stream, using <c>Stream.Write</c>.
    /// </summary>
    /// <param name="stream">Stream to write the buffer to.</param>
    /// <exception cref="ArgumentNullException">Stream invalid.</exception>
    /// <exception cref="ArgumentException">Used with T other than byte.</exception>
    public void WriteTo(Stream stream);
...
}

Usage example

The example demonstrates writing into a target stream that is itself wrapping a buffer writer, but in my local fork I am using it for another custom Stream.

using ArrayPoolBufferWriter<byte>? writerSource = new(), writerTarget = new();

/// populate the source buffer
Span<byte> spanSource = writerSource.GetSpan(4).Slice(0, 4);
byte[] data = { 1, 2, 3, 4 };
data.CopyTo(spanSource);
writerSource.Advance(4);

// Wrap the target writer into a custom internal stream type and produce a write-only
// stream that essentially mirrors the IBufferWriter<T> functionality as a stream.
using Stream stream = writerTarget.AsStream();

// zero-copy the internal buffer to the target stream
writerSource.WriteTo(stream);

Breaking change?

No

Alternatives

Currently to write to a given stream from a source buffer one may convert the written memory to array, incurring a buffer copy:

stream.Write(writerSource.WrittenMemory.ToArray(), offset: 0, count: writerSource.WrittenCount);

Additional context

No response

Help us help you

Yes, I'd like to be assigned to work on this item

[Sergio0694]

"Before the change, the only way to write out the underlying byte array to a stream requires a buffer copy."

Hey, thank you for the proposal! I'm not really sure I follow, could you not just do:

stream.Write(writerSource.WrittenSpan);

? 🤔

[Sergio0694]

Actually, if you're on .NET Standard 2.0, this approach will still do a copy (though it will use a pooled buffer). You can skip the copy entirely in one of two ways.

• Using a wrapper stream:

  writerSource.WrittenMemory.AsStream().CopyTo(stream);

• Extracting the array and doing a manual write:

  
  _ = MemoryMarshal.TryGetArray(writerSource.WrittenMemory, out ArraySegment<byte> segment);

stream.Write(segment.Array!, segment.Offset, segment.Count);

[vask-msft]

Doesn't this incur a buffer copy inside the public static void Write(this Stream stream, ReadOnlySpan<byte> buffer) though on my .NET standard 2.0 env?

[vask-msft]

Thanks a lot, the 2nd bullet (with TryGetArray) looks good, despite the additional wrapper layer through the extra Segment compared to the WriteTo suggestion. Yet adding the System.IO dependency to the ArrayPoolBufferWriter does feel wrong, so I'll change my client code to use TryGetArray instead.

[Sergio0694]

This made me realize ArrayPoolBufferWriter<T> should've also had DangerousGetArray() like the other types 🙂 I'm adding this in #616. Once that's in (will be in the 8.2 release), this would be simplified to:

ArraySegment<byte> segment = writerSource.DangerousGetArray();

stream.Write(segment.Array!, segment.Offset, segment.Count);