robinrodricks / FluentStorage

A polycloud .NET cloud storage abstraction layer. Provides Blob storage (AWS S3, GCP, FTP, SFTP, Azure Blob/File/Event Hub/Data Lake) and Messaging (AWS SQS, Azure Queue/ServiceBus). Supports .NET 5+ and .NET Standard 2.0+. Pure C#.
MIT License
263 stars 33 forks source link

[Bug] WriteFileAsync will not reduce file size resuling in data leak (SFTP) #52

Closed beeradmoore closed 7 months ago

beeradmoore commented 7 months ago

If you have a file locally that contains the following text

12345678

and you upload that file with await myRemoteStorage.WriteFileAsync(myRemoteFile, myLocalFile) you will create a file at myRemoteFile and it will be 8 bytes long.

If you now change the file locally to be

0000

and call await myRemoteStorage.WriteFileAsync(myRemoteFile, myLocalFile) then the result file on the server will not change its filesize and it will actually contain the text

00005678

I believe this comes from the WriteAsync method which opens a local and remote stream (see FluentFtpBlobStorage and SshNetSftpBlobStorage) and calls CopyAsync. This does exactly that, copies the stream of data across. But after the rest of the file is not removed.

This is very similar to the behaviour which casued people to be able to uncrop screenshots due to a bug in the image cropper.

Using UploadFile from the respective clients should solve this problem, but it prevents being able to append to a file.

I'll try it out and see how it goes, if I find a solution I'll create a PR.

Provider Impacted (may not be accurate)
FluentStorage
FluentStorage.AWS
FluentStorage.GCP
FluentStorage.Databricks
FluentStorage.FTP
FluentStorage.SFTP
FluentStorage.Azure.Blobs
FluentStorage.Azure.Files
FluentStorage.Azure.EventHub
FluentStorage.Azure.ServiceBus
FluentStorage.Azure.KeyVault
FluentStorage.Azure.ServiceFabric
FluentStorage.Azure.Queues
FluentStorage.Azure.DataLake

Skimming through code to see what is and isn't impacted. May not be 100% accurate. (EDIT: Tested FluentStorage.FTP thismroning and it does not have this problem)

Eyeballing the test here (without actuallying knowing how these tests work), changing the attributes to have something smaller after the longer test should find it? (assuming upload to the same location)

[InlineData("sample")]
[InlineData("123")]
[InlineData("123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123123")]
[InlineData("456")]
beeradmoore commented 7 months ago

Upon testing today I was not able to replicate in FluentStorage.FTP. I guess the underlying FtpDataStream object handles this correctly and the issue may only be with how SftpFileStream handles things.

It appears adding a .SetLength( fixes the issue.

robinrodricks commented 7 months ago

Fixed by OP.