Kubernetes pods considers requests value rather than limits values.

[thapabishwa]

I have deployed a k8s pod with both automemlimit and automaxproc and I have set the resources to the following:

resources:
  limits:
    cpu: 2
    memory: 4Gi

  requests:
     cpu: 1
     memory: 2Gi

automemlimits uses the request value rather than the limits values set on the k8s manifest. Is this expected behavior?

automaxprocs uses the limits value set on manifest rather than the request value.

I was under the impression that automemlimits did the same thing for memory limits like automaxprocs did for cpu limits.

Am I missing something here?

[KimMachineGun]

It is supposed to work with limits not requests, and in the EKS environments, it works as intended.

Can you provide some information about your k8s environment? If possible logs with AUTOMEMLIMIT_DEBUG=true env var would be helpful as well.

[thapabishwa]

I'm running my tests on minikube. My machine itself is a M2 mac-book pro. I'm using https://github.com/abiosoft/colima as my docker runtime.

This is the output of once I set AUTOMEMLIMIT_DEBUG=true on that container.

2023/11/09 03:37:10 GOMEMLIMIT=2076180480
{"level":"info","ts":"2023-11-09T03:37:10Z","msg":"maxprocs: Updating GOMAXPROCS=2: determined from CPU quota\n"}

The resource limits and requests are set to this like I previously discussed.

I'm using automemlimit by importing it directly.

import _ "github.com/KimMachineGun/automemlimit"

My base image is for the testing environment isgcr.io/distroless/static:debug and for the prod environment is gcr.io/distroless/static:static.

Could it be related to the base image that I'm currently using?

[thapabishwa]

Output of my docker info if it helps.

$ docker info

Client: Docker Engine - Community
 Version:    24.0.7
 Context:    colima
 Debug Mode: false

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 0cae528dd6cb557f7201036e9f43420650207b58
 runc version: 860f061b76bb4fc671f0f9e900f7d80ff93d4eb7
 init version:
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 6.1.55-0-virt
 Operating System: Alpine Linux v3.18
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 15.6GiB
 Name: colima
 ID: 19e4e118-0b6e-460f-a5c8-140e587e2201
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

[KimMachineGun]

It seems like another limit was used. Is there enough memory allocated for minikube? (FYI. 2,200 MiB is identical to minkube's default memory config)

// for some safety reasons, automemlimit uses 90% of the limit.
2,076,180,480 bytes / 0.9 in MiB = 2,200 MiB

[thapabishwa-plerionaut]

I think this is happening because I run minikube start without any manual intervention on CPU and memory limits. I'll try to increase the limits and run this again. Thanks for raising this.

[thapabishwa]

I can confirm that It works fine on EKS.

@KimMachineGun Have you tested if it works correctly on AKS and GKE?

[KimMachineGun]

Yes, I have tested EKS and GKE on every release and confirmed they worked correctly. Since I do not actively use Azure, AKS is not tested on every release; it is tested for changes related to cgroups.