S3 Client signature error on s3 object keys with special characters

[qxconverse]

Describe the bug

When using the regular S3 PUT command, this error is encountered if the object key contains a special character '@'

Expected Behavior

not response a signature error

Current Behavior

The request signature we calculated does not match the signature you provided. Check your key and signing method.

Reproduction Steps

#include <aws/core/Aws.h>
#include <aws/core/auth/AWSCredentialsProvider.h>
#include <aws/s3/S3Client.h>
#include <aws/s3/model/PutObjectRequest.h>
#include <iostream>
#include <fstream>
#include <sys/stat.h>

Aws::SDKOptions options;
    options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
    Aws::InitAPI(options);

    Aws::Client::ClientConfiguration cfg;
    cfg.endpointOverride = endpoint;
    cfg.scheme = Aws::Http::Scheme::HTTP;
    cfg.verifySSL = false;

    Aws::Auth::AWSCredentials cred(ak, sk);
    S3Client client(cred, cfg, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, false);

    Aws::S3::Model::PutObjectRequest object_request;
    object_request.WithBucket(s3_bucket_name).WithKey(s3_object_name);

    const shared_ptr<Aws::IOStream> input_data =
        Aws::MakeShared<Aws::FStream>("SampleAllocationTag",
                                      file_name.c_str(),
                                      ios_base::in | ios_base::binary);
    object_request.SetBody(input_data);

    auto response = client.PutObject(object_request);
    if (!response.IsSuccess())
    {
        auto error = response.GetError();
        cout << "PutObject ERROR: " << error.GetExceptionName() << ": " << error.GetMessage() << endl;
    }
    else
    {
        cout << "PutObject file: " << file_name << " upload to " << s3_bucket_name << " saved as " << s3_object_name << endl;
    }

    Aws::ShutdownAPI(options);
    return 0;

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

using git clone the latest version

Compiler and Version used

g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

Operating System and version

Ubuntu 20.04.1 LTS x86_64

[SergeyRyabinin]

Hi @qxconverse , Thank you for submitting the issue. For historical / legacy reasons, (initially) S3 and this SDK were not fully compliant with RFC URL encoding, resulting in a difference between the HTTP path and path used to compute the SigV4 signature.

It seems that you are overriding the service endpoint. I assume that you use some 3-rd party S3 server implementation, in that case you could also enable RFC-compliant encoding via setting options.compliantRfc3986Encoding = true or SetCompliantRfc3986Encoding(true). We've heard that this settings helps with other S3-like API providers.

We are also looking into ways how to fully move to a always compliant RFC encoding by default.

Best regards, Sergey

[jmklix]

I attempted to reproduce this error, but all special characters marked as "might require special handling" (&$@=;/:+ ,?) work with this sdk when not overriding the service endpoint. There might be some edge cases, but if you run into those please point them out to us. Can you modify your sample to reproduce this while using AWS S3 buckets?

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.