Closed thapabishwa closed 11 months ago
It is supposed to work with limits not requests, and in the EKS environments, it works as intended.
Can you provide some information about your k8s environment? If possible logs with AUTOMEMLIMIT_DEBUG=true
env var would be helpful as well.
I'm running my tests on minikube
. My machine itself is a M2 mac-book pro. I'm using https://github.com/abiosoft/colima as my docker runtime.
This is the output of once I set AUTOMEMLIMIT_DEBUG=true
on that container.
2023/11/09 03:37:10 GOMEMLIMIT=2076180480
{"level":"info","ts":"2023-11-09T03:37:10Z","msg":"maxprocs: Updating GOMAXPROCS=2: determined from CPU quota\n"}
The resource limits and requests are set to this like I previously discussed.
I'm using automemlimit
by importing it directly.
import _ "github.com/KimMachineGun/automemlimit"
My base image is for the testing environment isgcr.io/distroless/static:debug
and for the prod environment is gcr.io/distroless/static:static
.
Could it be related to the base image that I'm currently using?
Output of my docker info if it helps.
$ docker info
Client: Docker Engine - Community
Version: 24.0.7
Context: colima
Debug Mode: false
Server:
Containers: 2
Running: 2
Paused: 0
Stopped: 0
Images: 1
Server Version: 23.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 0cae528dd6cb557f7201036e9f43420650207b58
runc version: 860f061b76bb4fc671f0f9e900f7d80ff93d4eb7
init version:
Security Options:
seccomp
Profile: builtin
Kernel Version: 6.1.55-0-virt
Operating System: Alpine Linux v3.18
OSType: linux
Architecture: aarch64
CPUs: 8
Total Memory: 15.6GiB
Name: colima
ID: 19e4e118-0b6e-460f-a5c8-140e587e2201
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
It seems like another limit was used. Is there enough memory allocated for minikube? (FYI. 2,200 MiB is identical to minkube's default memory config)
// for some safety reasons, automemlimit uses 90% of the limit.
2,076,180,480 bytes / 0.9 in MiB = 2,200 MiB
I think this is happening because I run minikube start
without any manual intervention on CPU and memory limits.
I'll try to increase the limits and run this again.
Thanks for raising this.
I can confirm that It works fine on EKS.
@KimMachineGun Have you tested if it works correctly on AKS and GKE?
Yes, I have tested EKS and GKE on every release and confirmed they worked correctly. Since I do not actively use Azure, AKS is not tested on every release; it is tested for changes related to cgroups.
I have deployed a k8s pod with both
automemlimit
andautomaxproc
and I have set the resources to the following:automemlimits
uses the request value rather than the limits values set on the k8s manifest. Is this expected behavior?automaxprocs
uses the limits value set on manifest rather than the request value.I was under the impression that
automemlimits
did the same thing for memory limits likeautomaxprocs
did for cpu limits.Am I missing something here?