Logger write s3, DataIn or DataOut is binary data

[lvhuat]

Describe the bug

When i do s3 object writing. open the logger, it output something i don't expect:

[2023-03-03 03:07:43.722] [info] AwsSdk | *Debug* (HeaderIn) x-amz-request-id: 74YE1D4KDD18G374
[2023-03-03 03:07:43.722] [info] AwsSdk | *Debug* Returned http response code 200
[2023-03-03 03:07:43.722] [info] AwsSdk | *Debug* Cleaning up 0x561d524418a0
[2023-03-03 03:07:43.238] [info] AwsSdk | *Debug* (HeaderIn) x-amzn-RequestId: c7ae12c6-ed0d-437e-a268-7f18e29b7eb1
[2023-03-03 03:07:43.239] [info] AwsSdk | *Debug* Returned http response code 200
[2023-03-03 03:07:43.239] [info] AwsSdk | *Debug* Note: Http payloads are not being signed. signPayloads=0 http scheme=https
[2023-03-03 03:07:43.240] [info] AwsSdk | *Debug* Attempting to acquire curl connection.
[2023-03-03 03:07:43.253] [info] AwsSdk | *Debug* (Text) TCP_NODELAY set
[2023-03-03 03:07:43.418] [info] AwsSdk | *Debug* (SSLDataOut) 512bytes
[2023-03-03 03:07:43.508] [info] AwsSdk | *Debug* (Text) TLSv1.2 (IN), TLS handshake, Server key exchange (12):
[2023-03-03 03:07:43.509] [info] AwsSdk | *Debug* (SSLDataOut) 5bytes
[2023-03-03 03:07:43.511] [info] AwsSdk | *Debug* (Text) TLSv1.2 (IN), TLS handshake, Finished (20):
[2023-03-03 03:07:43.511] [info] AwsSdk | *Debug* (Text)  subjectAltName: host "bu-prod01.s3.us-west-2.amazonaws.com" matched cert's "*.s3.us-west-2.amazonaws.com"
[2023-03-03 03:07:43.708] [info] AwsSdk | *Debug* (DataOut) ���׷��ϯgb���i3 �5!��Q ���]ǘ[A�'~|w��Iq4������EX����_\^]`�N��`��Q0 F�(�`��Q0 F�(�*����a����(o01;31mAwsSdk | *Debug* (HeaderIn) Server: AmazonS3
[2023-03-03 03:07:43.722] [info] AwsSdk | *Debug* Request returned successful response.
[2023-03-03 03:07:43.723] [info] AwsSdk | *Debug* Cleaning up 0x561d526142c0
[2023-03-03 03:07:43.238] [info] AwsSdk | *Debug* (SSLDataIn) 5bytes

Expected Behavior

Print normal result or just print the size of the data.

Current Behavior

always happend

Reproduction Steps

    Aws::SDKOptions options;
    options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
    options.loggingOptions.logger_create_fn = [] {
      return std::make_shared<Aws::Utils::Logging::ConsoleLogSystem>(
          Aws::Utils::Logging::LogLevel::Trace);
    };
    Aws::InitAPI(options);

I use spdlog print log,use ConsoleLogSystem can also reproduce the issue.

  void save(const std::string& object_key,
            const std::shared_ptr<std::iostream>& body) {
    Aws::Client::ClientConfiguration config;
    config.requestTimeoutMs = 60000;
    config.connectTimeoutMs = 10000;
    Aws::S3::S3Client s3_client(config);
    Aws::S3::Model::PutObjectRequest request;
    request.SetBucket(_aws_s3_bucket);
    request.SetKey(object_key);
    request.SetBody(body);
    body->seekg(0, std::ios_base::end);

    body->exceptions(std::ios_base::goodbit);
    spdlog::info("s3::save {} {} start size={} bytes", _aws_s3_bucket,
                 object_key, body->tellg());
    body->seekg(0, std::ios_base::beg);

    Aws::S3::Model::PutObjectOutcome outcome = s3_client.PutObject(request);

    if (!outcome.IsSuccess()) {
      auto err = outcome.GetError();
      spdlog::error("s3::save failed code={} msg={} ", err.GetExceptionName(),
                    err.GetMessage());
      throw std::runtime_error("save snapshot failed," +
                               err.GetExceptionName() + "," + err.GetMessage());
    }

    spdlog::info("s3::save done:{}", object_key);
  }

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.8.184

Compiler and Version used

gcc (Ubuntu 7.5.0-6ubuntu2) 7.5.0

Operating System and version

ubuntu 20.04 LTS

[AlpenBauer]

Hi, I am having the same issue that the SDK log file is too large because of the binary data in the http response. Is there a way to solve this? Thanks

[DmitriyMusatkin]

it now only logs the size after the below change. https://github.com/aws/aws-sdk-cpp/commit/64e1d2e96004d1f705d1f8dfb216321807a6b934

[jmklix]

@lvhuat can you update to the latest version of this sdk and see if you have any other concerns with what is written to the logs?

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.