robinrodricks / FluentStorage

A polycloud .NET cloud storage abstraction layer. Provides Blob storage (AWS S3, GCP, FTP, SFTP, Azure Blob/File/Event Hub/Data Lake) and Messaging (AWS SQS, Azure Queue/ServiceBus). Supports .NET 5+ and .NET Standard 2.0+. Pure C#.
MIT License
263 stars 33 forks source link

Implementation of `IBlobStorage.WriteAsync` does not honor the `append` hint value in `FluentFtpBlobStorage` #23

Closed candoumbe closed 1 year ago

candoumbe commented 1 year ago

The current implementation of IBlobStorage.WriteAsyncdoes not honor the append hint value in FluentStorage.FTP implementation


        public async Task WriteAsync(string fullPath, Stream dataStream,
           bool append = false, CancellationToken cancellationToken = default) {
            AsyncFtpClient client = await GetClientAsync().ConfigureAwait(false);

            await retryPolicy.ExecuteAsync(async () => {
                using (Stream dest = await client.OpenWrite(fullPath, FtpDataType.Binary, true).ConfigureAwait(false)) {
                    await dataStream.CopyToAsync(dest).ConfigureAwait(false);
                }
            }).ConfigureAwait(false);
        }

I think the stream should be created using client.OpenAppend when the parameter append is true and client.OpenWrite otherwise.

candoumbe commented 1 year ago

@robinrodricks I started working on fixing this bug : https://github.com/robinrodricks/FluentStorage/compare/develop...candoumbe:FluentStorage:hotfix/ftpblobstorage-should-append-or-write-depending-on-append-parameter-value

Unfortunately, I currently encounter an issue when using TestContainers to setup a FTP fixture (using pure-ftpd docker image) but for now the test fails.

The test passes if I setup a container for the ftp server with a docker compose like this:

  ftpd_server:
    image: stilliard/pure-ftpd
    container_name: pure-ftpd
    ports:
      - "21:21"
      - "30000-30059:30000-30059"
    volumes: # remember to replace /folder_on_disk/ with the path to where you want to store the files on the host machine
      - "/home/<my folder>/data:/home/k2/data:rw"
      - "/home/<my folder>/passwd:/etc/pure-ftpd/passwd"
# uncomment for ssl/tls, see https://github.com/stilliard/docker-pure-ftpd#tls
#      - "/folder_on_disk/ssl:/etc/ssl/private/"
# or ssl/tls with Let's Encrypt (cert and key as two files)
#      - "/etc/letsencrypt/live/<your_server>/cert.pem:/etc/ssl/private/pure-ftpd-cert.pem"
#      - "/etc/letsencrypt/live/<your_server>/privkey.pem:/etc/ssl/private/pure-ftpd-key.pem"
    environment:
      PUBLICHOST: "localhost"
      FTP_USER_NAME: k2
      FTP_USER_PASS: k2
      FTP_USER_HOME: /home/user/
      FTP_MAX_CLIENTS: 50
# also for ssl/tls:
#      ADDED_FLAGS: "--tls=2"
    restart: always
candoumbe commented 1 year ago

@robinrodricks As the PR #25 was merged, when could we expect the hotfix to be available ?

robinrodricks commented 1 year ago

I'll do you one better. All released!

candoumbe commented 1 year ago

Thanks @robinrodricks !!