minio / minio-dotnet

MinIO Client SDK for .NET
https://docs.min.io/docs/dotnet-client-quickstart-guide.html
Apache License 2.0
575 stars 230 forks source link

High memory usage for PutObjectAsync with Streams #1138

Open mgj opened 4 months ago

mgj commented 4 months ago

Possibly related issues: #564 #710

Passing a Stream to PutObjectAsync appears to read the entire stream into memory causing memory usage to spike when uploading large files:

Screenshot 2024-07-20 153715

This memory is not freed when the PutObjectAsync call returns nor when the stream is disposed. Sequential uploads causes the memory usage to grow further. The memory does seem to be released later.

The call used for testing:

await _minioClient.PutObjectAsync(new PutObjectArgs()
    .WithContentType(contentType)
    .WithBucket(bucketName)
    .WithObject(objectName)
    .WithStreamData(blobStream)
    .WithObjectSize(contentLength)
);

Is there any way to make Stream uploading more efficient? Is there any way to control the memory usage?

Thanks in advance.