Closed krishnap80 closed 1 month ago
martincostello
commented
1 month ago Have you considered upgrading to .NET 8 and seeing if that improves the situation?
.NET 6 goes out of support in November so the chances of any memory leak being investigated, found and fixed in the remaining 2 patch releases isn't great.
En3Tho
commented
1 month ago Have you tried taking memory dumps and analyzing them? Requests per second doesn't really matter here - either you have a leak or you don't. Higher load will just make leakage more apparent
krishnap80
commented
1 month ago Thanks all for the reply. We will update to .net 8 in a few weeks. I will check after that and if needed will get back to you
On Sat, Aug 31, 2024 at 12:15 PM Igor Bagdamyan @.***> wrote:
Have you tried taking memory dumps and analyzing them? Requests per second doesn't really matter here - either you have a leak or you don't. Higher load will just make leakage more apparent
— Reply to this email directly, view it on GitHub https://github.com/dotnet/aspnetcore/issues/57638#issuecomment-2322975155, or unsubscribe https://github.com/notifications/unsubscribe-auth/BK5SW6CKX4ZTIZJVXBYK62TZUH23JAVCNFSM6AAAAABNOANCXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRSHE3TKMJVGU . You are receiving this because you authored the thread.Message ID: @.***>
amcasey
commented
1 month ago To be able to help with a memory leak, we'd need a repro app (usually infeasible) or, at least, the name of the type that's leaking. Can you please file a new issue if you continue to have problems in .net 8?
Is there an existing issue for this?
Describe the bug
We have our API running in .NET 6 running in docker container deployed in AWS ECS Fargate. We are running 12 containers of 8 GB size each. All the containers combined process around 20 requests/sec in total. What I am seeing is the memory consumption keeps on increasing in all the containers until they reach 98% and then eventually create a production alert and incident. Then the devops person has to recycle the containers by redeploying the version again. I checked in code if we have any memory leaks and checked and fixed as far as I could. We are connecting to postgres using Npgsql and dapper and we are having using for each db call. We don't have any other unmanaged memory being used and all are managed data types like lists, strings etc.
I have been facing this issue for several months and researched on this issue and observed in other groups that they suggested to use servergarbagecollection as false to use workstation garbage collection. I tried that and here it is much lower memory consumption compared to server GC but still the memory increase happens but in a slower manner. When using server GC the memory reaches > 98% in 2-3 weeks and in workstation garbage collection it takes more than 1.5 months to reach that stage.
The next thing I tried is to set a hard memory limit in the container. Kept around 80% of the container size that is 7GB as hard memory limit. In this case, things became worse as the .NET is throwing out of memory exception when the memory reaches grows to near by the hard limit and the APIs start failing. Therefore we have removed the hard limit.
I searched on this in lot of places in internet and could not find any working solution for my issue. We have other APIs that have less call rate < 5-10 requests/sec which don't have this issue. Please suggest if this is because of high request rate and on what could be the fix here. Thanks in advance for your valuable inputs.
Expected Behavior
Memory consumption should remain constant and if the memory limit is about to be reached GC should automatically reduce the consumed memory to keep it within limits
Steps To Reproduce
Nothing special, just the application keeps on running in production
Exceptions (if any)
None
.NET Version
6.0.32
Anything else?
No response