Closed huteshmahajan closed 2 days ago
This issue is related to #2706
Hi @huteshmahajan ,
Deploy an application which uses aws/aws-sdk-go-v2 wih high volume of data in s3
This is too broad of reproduction conditions for us to take action on. Can you please provide the code you used to test this?
Thanks, Ran~
Hello @RanVaknin This is our production code. So cannot provide actual code snippet. But, I'd like to provide some insights into the issue we're facing. We invoke multiple AWS Lambda functions concurrently from our appplication. In some cases, lambda response exceeds the AWS Lambda Service's standard limit of 6MB and that after retries, leads to OOM.
Hi @huteshmahajan ,
I'm are not after your business logic. All I'm after is a reproducible code snippet that can reliably raise the error you are seeing. In the issues you linked the issue was described with S3, and was unrelated to Lambda.
We invoke multiple AWS Lambda functions concurrently from our appplication. In some cases, lambda response exceeds the AWS Lambda Service's standard limit of 6MB and that after retries, leads to OOM.
This is still too broad of a description. Is your current application running on lambda itself? What do you mean by: "lambda response exceeds the AWS Lambda Service's standard limit of 6MB"? The response from which lambda?
What we are after is a code snippet / example repository that can reliably demonstrate this issue so we may reproduce it ourselves and investigate the root cause.
Thanks, Ran~
This issue has not received a response in 1 week. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.
Acknowledgements
go get -u github.com/aws/aws-sdk-go-v2/...
)Describe the bug
We are encountering an Out Of Memory (OOM) issue in our application when attempting to retrieve data using AWS Lambda. Upon investigation, we've identified that the aws/aws-sdk-go-v2 library is consuming a significant amount of memory, leading to the pod being terminated in an OOM killed state.
Observations
Screenshots Attach the relevant screenshots from Pyroscope showing the memory consumption for the inuse-space memory.
Expected Behavior
The aws/aws-sdk-go-v2 library should manage memory efficiently, preventing the application from reaching an OOM state.
Current Behavior
The application consistently enters an OOM killed state when using the aws/aws-sdk-go-v2 library, indicating potential issues with memory management within the library.
Reproduction Steps
Deploy an application which uses aws/aws-sdk-go-v2 wih high volume of data in s3 Monitor memory usage using a profiling tool (e.g., Pyroscope). Observe that memory consumption spikes significantly during data retrieval, eventually leading to the pod being killed due to OOM.
Possible Solution
No response
Additional Information/Context
No response
AWS Go SDK V2 Module Versions Used
github.com/aws/aws-sdk-go-v2 v1.24.1 github.com/aws/aws-sdk-go-v2/config v1.26.6 github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.15 github.com/aws/aws-sdk-go-v2/service/lambda v1.49.7 github.com/aws/aws-sdk-go-v2/service/s3 v1.48.1 github.com/aws/smithy-go v1.19.0
Compiler and Version used
go version go1.22.3 darwin/arm64
Operating System and version
MacOs 14.1.1 (23B81)