InitAPI memory leak

[saad-alsaad1]

Describe the bug

I've compiled AWS SDK on three RHEL machines. RHEL7, RHEL8, and RHEL9 I noticed that when I call Aws::InitAPI(options) the memory raise by 1847 MB on RHEL7 and RHEL9! while on RHEL8 Aws::InitAPI(options) API consume around 370 MB. Also, when I call the Aws::ShutdownAPI(options) function it only free 50 MB. I've the following library versions on each machine:

RHEL7: Cmake: 3.14.2 GCC: 8.3.1 libz: 1.2.7 libcurl: 7.29.0 OpenSSL & libcrypto: 1.0.2k Python: 3.6.8

RHEL8: Cmake: 3.14.2 GCC: 8.5.0 libz: 1.2.11 libcurl: 7.61.1 OpenSSL & libcrypto: 1.1.1k Python: 3.6.8

RHEL9: Cmake: 3.26.5 GCC: 11.4.1 libz: 1.2.11 libcurl: 7.76.1 OpenSSL & libcrypto: 3.0.7 Python: 3.9.18

I used the following command to compile AWS SDK on the machines:

./aws-sdk-cpp -DCMAKE_BUILD_TYPE=Release -DCURL_INCLUDE_DIR=/usr/include/ -DBUILD_ONLY="s3;sts;s3-crt;ec2;core;ec2-instance-connect;ecr;ecs;ecr-public;s3control;transfer;iam;identity-management;access-management;monitoring;s3-encryption" -DCMAKE_PREFIX_PATH=./aws-sdk-libs -DCMAKE_INSTALL_PREFIX=./aws-sdk-libs -DBUILD_SHARED_LIBS=ON

Code:

    Aws::SDKOptions options;
    switch(log_level){
        case 1:
            options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Fatal;
        break;
        case 2:
            options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Error;
        break;
        case 3:
            options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Warn;
        break;
        case 4:
            options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Info;
        break;
        case 5:
            options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Debug;
        break;
        case 6:
            options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
        break;
    }

    Aws::InitAPI(options); 

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

Expected Behavior

I believe Aws::InitAPI shouldn't need around 1.8 GB of memory on Centos 7 and 9 since on Centos 8 it only consume 370 MB of memory. I guess Aws::ShutdownAPI should clean all the memory allocated by Aws::InitAPI.

Current Behavior

Aws::InitAPI(options) take 1847 MB of memory on RHEL7 and RHEL9, but on RHEL8 it only take 370 MB. Aws::ShutdownAPI(options) doesn't free all the memory allocated by Aws::InitAPI(options).

Reproduction Steps

Compile AWS SDK on RHEL 7.6 or RHEL 9.4 with the follow libs: RHEL7: Cmake: 3.14.2 GCC: 8.3.1 libz: 1.2.7 libcurl: 7.29.0 OpenSSL & libcrypto: 1.0.2k Python: 3.6.8

RHEL9: Cmake: 3.26.5 GCC: 11.4.1 libz: 1.2.11 libcurl: 7.76.1 OpenSSL & libcrypto: 3.0.7 Python: 3.9.18

Use the following command for compilation: ./aws-sdk-cpp -DCMAKE_BUILD_TYPE=Release -DCURL_INCLUDE_DIR=/usr/include/ -DBUILD_ONLY="s3;sts;s3-crt;ec2;core;ec2-instance-connect;ecr;ecs;ecr-public;s3control;transfer;iam;identity-management;access-management;monitoring;s3-encryption" -DCMAKE_PREFIX_PATH=./aws-sdk-libs -DCMAKE_INSTALL_PREFIX=./aws-sdk-libs -DBUILD_SHARED_LIBS=ON

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.365

Compiler and Version used

gcc 8.3.1 & gcc 8.5.0 & gcc 11.4.1

Operating System and version

Centos 7.6 & Centos 8.5.2 & Centos 9.4

[jmklix]

I'm not seeing any memory leaks when reproducing this on RHEL9.4.

#include <iostream>
#include <aws/core/Aws.h>

using namespace std;
using namespace Aws;

int main() {
    Aws::SDKOptions options;
    options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;

    Aws::InitAPI(options);
    {
            std::cout << "Test" <<std::endl;
    }
    Aws::ShutdownAPI(options);
    return 0;
}

Can you try the following:

• make sure you are following the basic guidelines of making all sdk calls within {}
• make sure you are only calling InitAPI and ShutdownAPI once each
• run valgrind memory check with your application.

Valgrind also put my simple application usage at only 15 MB. So it would be helpful if you could give us a minimal repro code sample that shows a memory leak.

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

[saad-alsaad1]

I used your example code and tried Valgrind on centos 9.4, it didn't help. I noticed in the trace log that Initializing edge-triggered epoll is redundant 24 times after Successfully reloaded configuration, I think this is not normal.

Here is a snapshot: