ClientConfiguration.scheme is not working

[walterchenqq]

Describe the bug

Setting ClientConfiguration.scheme=HTTP and then PutObject, underlying connection is still using HTTPS endpoint.

Expected Behavior

According to the manual , setting ClientConfiguration.scheme = HTTP to specify the URI addressing scheme, and underlying connection is established using HTTP.

Current Behavior

Reproduction Steps

#include <aws/s3/S3Client.h>
#include <aws/core/auth/AWSCredentialsProviderChain.h>
#include <aws/core/auth/AWSCredentials.h>
#include <aws/s3/S3EndpointProvider.h>
#include <aws/s3/model/PutObjectRequest.h>
#include <aws/s3/model/PutObjectResult.h>
using namespace Aws;
using namespace Aws::Auth;

bool putObject(const Aws::String &bucketName,
        const Aws::String &fileName,
        const Aws::S3::S3ClientConfiguration &clientConfig) {

    Aws::Auth::AWSCredentials credential("${TEST_SECRET_ID}", "${TEST_SECRET_KEY}");
    Aws::S3::S3Client s3Client(credential, std::make_shared<Aws::S3::S3EndpointProvider>(), clientConfig);

    Aws::S3::Model::PutObjectRequest request;
    request.SetBucket(bucketName);
    //We are using the name of the file as the key for the object in the bucket.
    //However, this is just a string and can be set according to your retrieval needs.
    request.SetKey(fileName);

    std::shared_ptr<Aws::IOStream> inputData =
        Aws::MakeShared<Aws::FStream>("SampleAllocationTag",
                fileName.c_str(),
                std::ios_base::in | std::ios_base::binary);

    if (!*inputData) {
        std::cerr << "Error unable to read file " << fileName << std::endl;
        return false;
    }

    request.SetBody(inputData);

    Aws::S3::Model::PutObjectOutcome outcome =
        s3Client.PutObject(request);

    if (!outcome.IsSuccess()) {
        std::cerr << "Error: putObject: " <<
            outcome.GetError().GetMessage() << std::endl;
    } else {
        std::cout << "Added object '" << fileName << "' to bucket '"
            << bucketName << "'.";
    }

    return outcome.IsSuccess();
}

int main(int argc, char **argv) {
    Aws::SDKOptions options;
    // Optionally change the log level for debugging.
    options.loggingOptions.logLevel = Utils::Logging::LogLevel::Trace;
    Aws::InitAPI(options); // Should only be called once.
    int result = 0;
    {
        Aws::Client::ClientConfiguration clientConfig;
        clientConfig.region = "us-east-2";
        clientConfig.scheme = Aws::Http::Scheme::HTTP;

        putObject("${TEST_BUCKET}", argv[1], clientConfig);
    }

    Aws::ShutdownAPI(options); // Should only be called once.
    return result;
}

Compile and run this code: ./my_put_object /path/to/file check the underlying endpoint: cat awssdk*.log |grep "evaluated the endpoint"

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

Version:1.11.405

Compiler and Version used

gcc (GCC) 4.8.5

Operating System and version

Linux 3.10.107

[jmklix]

I'm not able to reproduce this locally using the code you gave above. Can you run this in a docker container and reproduce the same error of http payloads not being signed?

[walterchenqq]

The payload was signed indeed! But I was expecting the underlying connection being established via HTTP(not HTTPS) by this setting:

[walterchenqq]

For short, I just want HTTP connection being established in my case, NOT HTTPS.

[sbera87]

The logs stem from AWSAuthV4Signer::SignRequestWithCreds when this logical block is not satisfied

if(signBody || request.GetUri().GetScheme() != Http::Scheme::HTTPS)
{
    ...
}

which means signBody is false (its likely that signing policy set to PayloadSigningPolicy::Never. This comes from clientConfiguration in your constructor) and
endpoint resolution for put object request doesn't set the scheme in the uri (It could be due to put object request GetEndpointContextParams() is not set hence could be picking up endpoint params from the client configuration )

If possible can you please print the contents of client configuration, that can help our team replicate the issue

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

[walterchenqq]

The logs stem from AWSAuthV4Signer::SignRequestWithCreds when this logical block is not satisfied

if(signBody || request.GetUri().GetScheme() != Http::Scheme::HTTPS)
{
  ...
}

which means signBody is false (its likely that signing policy set to PayloadSigningPolicy::Never. This comes from clientConfiguration in your constructor) and endpoint resolution for put object request doesn't set the scheme in the uri (It could be due to put object request GetEndpointContextParams() is not set hence could be picking up endpoint params from the client configuration )

If possible can you please print the contents of client configuration, that can help our team replicate the issue

Have you run the example code I provided above?

[walterchenqq]

Setting this parameter just expects the underlying connection to be HTTP (but actually it is still HTTPS), but it seems that you have been convincing me that the underlying connection is indeed not HTTPS.

[sbera87]

I confirm the issue observed with the code presented. This is not the expected usage. The scheme field in client configuration is only used along with config.endpointOverride field. Default protocol used by s3 client is https. For your case, the scheme remains HTTPS because there is no endpoint override and the endpoint rules using default endpoint provider also results in resolution of the scheme as "HTTPS" from the url.

Can you please try using endpoint override and pass in the endpoint string you prefer along with the protocol

[walterchenqq]

I confirm the issue observed with the code presented. This is not the expected usage. The scheme field in client configuration is only used along with config.endpointOverride field. Default protocol used by s3 client is https. For your case, the scheme remains HTTPS because there is no endpoint override and the endpoint rules using default endpoint provider also results in resolution of the scheme as "HTTPS" from the url.

Can you please try using endpoint override and pass in the endpoint string you prefer along with the protocol

That is exactly the alternative I'm using now. But if you've already set the config.endpointOverride field to use HTTP, there's absolutely no need to set config.scheme。 Also, I don't see any instructions in the documentation on how to use these two fields together.

[sbera87]

"But if you've already set the config.endpointOverride field to use HTTP, there's absolutely no need to set config.scheme" -That is true. You don't have to set scheme in that case. The scheme override is to switch between schemes if your endpoint does support it.

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.