possible memory leak in K8s

[JHinW]

Q; we have lighthouse as seed services which works fine, however 500M memory is retained by each instance seems a strange thing, there must be some weakness cause this issue.

[JHinW]

A: process running under lovely docker can be inspected by donet-dump , refer to https://github.com/dotnet/diagnostics/blob/master/documentation/dotnet-dump-instructions.md

[JHinW]

in repo of akkadotnet there has been a lot of mem leak issues created there, certainly it may not be the one you ran into but could be a good reference： https://github.com/akkadotnet/akka.net/issues/3735

to kill a process in ubuntu linux: https://www.linux.com/training-tutorials/how-kill-process-command-line/

[JHinW]

working set makes the memory consumption keeping increasing: https://social.msdn.microsoft.com/Forums/en-US/442bd555-f9b5-41b5-a97f-3fcbdae7fd69/how-to-reduce-the-workingset-for-a-process

the key description of why app consume so much on app's startup:

The result of this is that on startup your app consumes a lot of resources that you will not need any longer once the app is up and running

https://blog.csdn.net/weixin_30609331/article/details/99590314

[JHinW]

unfortunately, memory still keeps increasing and not sure if it is normal, the problem seems come from dotnetty !!!

[JHinW]

another snapshot of memory usage which make me believe that it is an issue of workingset-allocating in OS:

[JHinW]

i will try set param namely ServerGarbageCollection as true by following the instruction from this link: https://docs.microsoft.com/en-us/dotnet/core/run-time-config/garbage-collector.

i will continue to monitor the effects of this setting!!!!!

garbage collection action executes too fast:

[JHinW]

eventually i realize that try to recycle memory pages allocated to one particular process is not a good choice, to make container running under limited resources : https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

[JHinW]

another question about memory allocation and recycle, how to invoke Dispose method for memory managing

with link below, you will get how finalize method is invoked during GC processing. https://stackoverflow.com/questions/13954829/gc-collect-and-finalize

[JHinW]

Another trial with version upgraded to 14.19 :
https://github.com/petabridge/lighthouse

after three days 👍

five days without restart:

i think new verision of lighthouse did a great garbage collection: (5 days after last deployment)