awslabs / aws-c-s3

C99 library implementation for communicating with the S3 service, designed for maximizing throughput on high bandwidth EC2 instances.
Apache License 2.0
93 stars 37 forks source link

CopyObjectRequest wrong host URL #424

Open jmklix opened 4 months ago

jmklix commented 4 months ago

original issue: https://github.com/aws/aws-sdk-cpp/issues/2822

Describe the bug

When I try to copy object from one folder to another in same bucket I get an error Invalid response status from request (aws-c-s3: AWS_ERROR_S3_INVALID_RESPONSE_STATUS). This happens only for CopyObject, for HeadObject, ListObjects, GetObject, PutObject everything is correct

Expected Behavior

Expected host URL should be s3.giraffe360-mimosa.com and file should be copied from /cold-data/projects/db9768d14f7c4055aa7518e42b633888/floorplan/roomsketcher/floorplan-ALL-final_1_0.svg to /cold-data/projects/db9768d14f7c4055aa7518e42b633888/floorplan/roomsketcher/final_backups/floorplan-ALL-final_1_0_2024-01-24_13-03-01.svg

Current Behavior

Based on logs I see it has wrong host URL

[INFO] 2024-01-24 11:03:01.675 AuthSigning [140735609304640] (id=0x7fff88004380) Signing successfully built canonical request for algorithm SigV4, with contents 
HEAD
/projects/db9768d14f7c4055aa7518e42b633888/floorplan/roomsketcher/floorplan-ALL-final_1_0.svg

host:cold-data.giraffe360-mimosa.com
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20240124T110301Z

host;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD

For other requests it is correct

[INFO] 2024-01-24 11:03:01.541 AuthSigning [140735592519232] (id=0x7fff840032d0) Signing successfully built canonical request for algorithm SigV4, with contents 
PUT
/cold-data/projects/db9768d14f7c4055aa7518e42b633888/floorplan/roomsketcher/geometry.json

content-length:4073
content-md5:rg4HTU7lw0cuZltmQKY+2g==
content-type:binary/octet-stream
host:s3.giraffe360-mimosa.com
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20240124T110301Z

content-length;content-md5;content-type;host;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD

cold-data is bucket name.

Reproduction Steps

  Aws::Auth::AWSCredentials credentials(access_key, secret_key, session_token);

  Aws::S3Crt::ClientConfiguration config;
  config.endpointOverride = host_;
  config.useVirtualAddressing = false;
  config.verifySSL = false;
  config.enableEndpointDiscovery = false;
  config.enableHostPrefixInjection = false;
  config.region = Aws::Region::US_EAST_1;
  config.scheme = Aws::Http::Scheme::HTTPS;
  config.disableMultiRegionAccessPoints = true;
  config.disableS3ExpressAuth = true;
  config.payloadSigningPolicy = Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never;
  config.useUSEast1RegionalEndPointOption = Aws::S3Crt::US_EAST_1_REGIONAL_ENDPOINT_OPTION::LEGACY;
  config.enableEndpointDiscovery = false;
  config.enableHostPrefixInjection = false;
  config.version = Aws::Http::Version::HTTP_VERSION_3;

  std::unique_ptr<Aws::S3Crt::S3CrtClient> client = std::make_unique<Aws::S3Crt::S3CrtClient>(credentials, config);

  Aws::S3Crt::Model::CopyObjectRequest request;
  request.SetBucket(bucket);
  request.SetKey(dst_path);
  request.SetCopySource(bucket + "/" + src_path);
  auto outcome = client->CopyObject(request);
  if (!outcome.IsSuccess()) {
    std::cerr << "Failed to copy file " << src_path << " to " << dst_path
               << " in bucket: " " << bucket
               << " with error: " << outcome.GetError().GetMessage()
               << " and error code: " << outcome.GetError().GetExceptionName();
  }

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.249

Compiler and Version used

gcc 13.1.0

Operating System and version

Ubuntu 22.04.3 LTS

jmklix commented 4 months ago

This is a problem with aws-c-s3. opening this issue here