microsoft / ApplicationInsights-Profiler-AspNetCore

Application Insights Profiler sample and documentation
MIT License
66 stars 23 forks source link

Memory % Calculation in Kubernetes #130

Closed ericmattingly closed 3 years ago

ericmattingly commented 3 years ago

I have the 2.2.0-beta6 profiler enabled on a .NET Core 3.1 service running in a Linux Alpine container in AKS. I have noticed that profiler is taking captures due to the default Memory Trigger (80%) but its not clear to me how its calculating the memory. On the "Configure Application Insights Profiler" blade its consistently saying the memory was ~90% every time a capture is performed (even the RandomSchedulingPolicy captures). My service is the only process in the container so I'm guessing that's why it's always considered around ~90% utilization?

.NET Core doesn't have a % memory utilization counter, only a working-set so I'm wondering if something can be improved here.

xiaomi7732 commented 3 years ago

The memory rate, on Linux, comes from /proc/meminfo. The formula is: (1-(free/total) * 100%):

For example, at the point of gathering, if the meminfo looks like it below:

saars@machine:~$ cat /proc/meminfo
MemTotal:       12993484 kB
MemFree:        12875448 kB
MemAvailable:  ...

The rate is (1-(12875448/12993484))* 100% = 0.91%.

In your case, you have utilized almost 90% of the memory of your application. You will need to decide whether that is because of a high utilization by design, or is there a potential memory leak - that GC happens constantly when under memory pressure.

Under the circumstance where memory utilize rate is constantly high and it is by design, you can set the criteria for memory trigger to be higher than the default of 80%, for example:

image