search

aws / aws-sdk-cpp

AWS SDK for C++
Apache License 2.0
1.96k stars 1.05k forks source link

DefaultRateLimiter get stuck indefinitely #2949

Closed qqwangxiaow closed 4 months ago

qqwangxiaow commented 4 months ago

Describe the bug

We have set a 100MB upload rate limit for the AWS S3 SDK, but sometimes the upload threads get stuck indefinitely.

config.writeRateLimiter = Aws::MakeShared<Aws::Utils::RateLimits::DefaultRateLimiter<>>("s3", 100 * 1024 * 1024);

stack

0 0x00007fca25e097fd in nanosleep () at ../sysdeps/unix/syscall-template.S:84

1 0x00000000005db796 in std::this_thread::sleep_for<long, std::ratio<1l, 1000l> > (__rtime=...) at /usr/include/c++/9/thread:378

2 Aws::Utils::RateLimits::DefaultRateLimiter<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1l> >, true>::ApplyAndPayForCost (this=this@entry=0x3dc4510, cost=614)

at external/_main~data_deps_ext~aws/aws-cpp-sdk-core/include/aws/core/utils/ratelimiter/DefaultRateLimiter.h:115

3 0x000000000084e9d1 in Aws::Http::CurlHttpClient::MakeRequestInternal(Aws::Http::HttpRequest&, std::shared_ptr&, Aws::Utils::RateLimits::RateLimiterInterface, Aws::Utils::RateLimits::RateLimiterInterface) const () at external/_main~data_deps_ext~aws/aws-cpp-sdk-core/source/http/curl/CurlHttpClient.cpp:461

4 0x0000000000851acb in Aws::Http::CurlHttpClient::MakeRequest(std::shared_ptr const&, Aws::Utils::RateLimits::RateLimiterInterface, Aws::Utils::RateLimits::RateLimiterInterface) const () at external/_main~data_deps_ext~aws/aws-cpp-sdk-core/source/http/curl/CurlHttpClient.cpp:719

5 0x000000000099a26c in Aws::Client::AWSClient::AttemptOneRequest(std::shared_ptr const&, Aws::AmazonWebServiceRequest const&, char const, char const) const ()

at /usr/include/c++/9/bits/char_traits.h:372

6 0x000000000099bf2e in Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const, char const) const ()

at external/_main~data_deps_ext~aws/aws-cpp-sdk-core/source/client/AWSClient.cpp:225

7 0x000000000099dc55 in Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const, char const) const ()

at external/_main~data_deps_ext~aws/aws-cpp-sdk-core/source/client/AWSClient.cpp:917

Expected Behavior

restart the work

Current Behavior

stuck forever

Reproduction Steps

low probability event

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.7.336

Compiler and Version used

gcc-9

Operating System and version

Debian GNU/Linux 9.8 (stretch)

DmitriyMusatkin commented 4 months ago

1.7.336 is about 4 years old and we dont support it anymore. I would recommend upgrading to the latest sdk and seeing if you can still repro it.

qqwangxiaow commented 4 months ago

It looks like the relevant code has not changed.

jmklix commented 4 months ago

Can you update to the latest version and see if you still get the errors? Also can you give us minimal reproduction code and logs from when it happens?

github-actions[bot] commented 4 months ago

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.