robinrodricks / FluentStorage

A polycloud .NET cloud storage abstraction layer. Provides Blob storage (AWS S3, GCP, FTP, SFTP, Azure Blob/File/Event Hub/Data Lake) and Messaging (AWS SQS, Azure Queue/ServiceBus). Supports .NET 5+ and .NET Standard 2.0+. Pure C#.
MIT License
226 stars 31 forks source link

ZipArchive backed IBlobStorage throws occasionally when calling WriteAsync #38

Open ckpearson opened 10 months ago

ckpearson commented 10 months ago

Sometimes when doing the following:

using var stream = ... // a Stream here
await blobStorage.WriteAsync("path_here", stream);

OR:

var stream = ... // a Stream here
await blobStorage.WriteAsync("path_here", stream);
await stream.DisposeAsync();

OR:

var bytes = ... // a byte array
await blobStorage.WriteAsync("path_here", bytes);

Occasionally a System.ObjectDisposedException: 'Cannot access a closed file.' will be raised.

This code path is being called by a background processor that utilises a worker pool, the idea being that files for writing are enqueued as tasks, and the pool will wait for all tasks to complete; so I'm wondering if there's a threading / async issue occurring here?

ckpearson commented 10 months ago

If I set my worker pool to only have a single worker (essentially emulating sequential processing) then the issue does disappear.

robinrodricks commented 5 months ago

Any fix and PR will be greatly appreciated.