search

docker / compose

Define and run multi-container applications with Docker
https://docs.docker.com/compose/
Apache License 2.0
33.98k stars 5.22k forks source link

`mem_limit` not working in Docker Compose on macOS #8772

Closed dhruvkb closed 2 years ago

dhruvkb commented 3 years ago

Description

Docker-Compose does not support unit sufix on mem_limit on macOS.

Steps to reproduce the issue:

Consider the docker-compose.yml file with a mem_limit.

version: '2.4'

services:
  es:
    mem_limit: '4g'

Running docker compose up raises the following error.

error while interpolating services.es.mem_limit: failed to cast to expected type: strconv.ParseInt: parsing "4g": invalid syntax

Describe the results you received: The aforementioned error is raised.

Describe the results you expected: The containers should be started with the prescribed mem_limit.

Additional information you deem important (e.g. issue happens only occasionally): This seems to work fine in Linux environments (e.g. GitHub Actions). It also works if a value without a unit suffix is provided (such as 4294967296 instead of 4g).

Output of docker compose version:

Docker Compose version v2.0.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.3)
  compose: Docker Compose (Docker Inc., v2.0.0)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 9
  Running: 9
  Paused: 0
  Stopped: 0
 Images: 45
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.841GiB
 Name: docker-desktop
 ID: WDJF:72CM:5OCW:46O3:F5VS:P5PS:EETY:RT5A:NTEP:DB3B:YL7D:HABP
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

Docker Desktop for Mac running on macOS Big Sur 11.6

Screenshot 2021-10-08 at 3 02 43 PM
ndeloof commented 3 years ago

I tested with compose v2.0.1

$ docker-compose version
Docker Compose version 2.0.1
$ docker-compose config
services:
  es:
    image: redis
    mem_limit: "4294967296"
    networks:
      default: null
networks:
  default:
    name: truc_default
ndeloof commented 3 years ago

Could you please give 2.0.1 a try to confirm this issue has been fixed ?

dhruvkb commented 3 years ago

@ndeloof I'm using Docker Desktop on macOS with the "Use Docker Compose v2" checkbox activated. The command docker compose version says v2.0.0. How do I update it?

KraProgrammer commented 3 years ago

I can reproduce the issue on Windows. Using version 2.0.1 does not help.

gauthierl commented 3 years ago

I can reproduce this issue on Linux too and with the parameter shm_size.

The issue is that the syntax with suffix (gb, g) is not working anymore. It's working if we convert to Int manually

error while interpolating services.postgres.shm_size: failed to cast to expected type: strconv.ParseInt: parsing "1g": invalid syntax

Docker compose version :

❯ docker-compose version
Docker Compose version 2.0.1
chrispyduck commented 2 years ago

Just observed using docker compose 2.1.0 and 2.1.1 on macOS: error while interpolating services.<name>.mem_limit: failed to cast to expected type: strconv.ParseInt: parsing "1400m": invalid syntax

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.3)
  compose: Docker Compose (Docker Inc., v2.1.1)
  scan: Docker Scan (Docker Inc., 0.9.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.10
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 11.7GiB
 Name: docker-desktop
 ID: Z2AD:7NGH:6KEG:67XR:MZGJ:6NSM:4UOI:S62V:FRRK:UYNS:ZY4O:EEIE
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
rherrick commented 2 years ago

I'm getting the same error with shm_size in docker-compose as @gauthierl. The following works with docker-compose v2.2.1:

shm_size: "256mb"

But this doesn't work:

# .env file
ENV_PG_SHM_SIZE=256mb
# docker-compose.yml
version: "3.7"

services:

  db:
    image: postgres:12-alpine
    shm_size: "${ENV_PG_SHM_SIZE}"

It fails with the error message:

error while interpolating services.xnat-vagrant-db.shm_size: failed to cast to expected type: strconv.ParseInt: parsing "256mb": invalid syntax

When I switch Use Docker Compose V2 off in Docker Desktop, I get docker-compose v1.29.2 and either approach works.

Current version info is:

iMac (Retina 5K, 27-inch, 2017) on OS X 11.6.1
Version 4.3.2 (72729)
Engine: 20.10.11
Compose: v2.2.1
Credential Helper: 0.6.4
Kubernetes: v1.22.4
Snyk: v1.801.0

That said, it's been happening for a while. Not sure when it started.

sagittarius-rbrisley commented 2 years ago

I get the same running the up.ps1 in this repo https://github.com/Sitecore/MVP-Site This is on windows. Two different errors "error while interpolating services.mssql.mem_limit: failed to cast to expected type: strconv.ParseInt: parsing "2GB": invalid syntax" and "error while interpolating services.cd.mem_limit: failed to cast to expected type: strconv.ParseInt: parsing "4GB": invalid syntax". WSL 2 engine enabled, "Use Docker Compose v2" is enabled. If I check Docker Compose is 2.2.3. Turning "Use Docker Compose v2" off and it then works.

Docker info `Client: Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc., v0.7.1) compose: Docker Compose (Docker Inc., v2.2.3) scan: Docker Scan (Docker Inc., v0.16.0)

Server: Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 1 Server Version: 20.10.12 Storage Driver: windowsfilter Windows: Logging Driver: json-file Plugins: Volume: local Network: ics internal l2bridge l2tunnel nat null overlay private transparent Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog Swarm: inactive Default Isolation: hyperv Kernel Version: 10.0 19044 (19041.1.amd64fre.vb_release.191206-1406) Operating System: Windows 10 Pro Version 2009 (OS Build 19044.1415) OSType: windows Architecture: x86_64 CPUs: 12 Total Memory: 31.94GiB Name: Chillrend ID: GKM3:IH57:KYFV:XX2B:DFOA:VQ6A:BRRD:WHNI:DKSB:72DV:MXNK:FVN4 Docker Root Dir: C:\ProgramData\Docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine`

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

dcorking commented 2 years ago

I can confirm that this bug exists in Docker Compose version v2.2.1 but appears to be fixed in Docker Compose version v2.12.0 (for me on Docker.app for macOS Intel 11.7 ) (The string I can now successfully assign from .env to mem_limit is 2g ).

stale[bot] commented 2 years ago

This issue has been automatically marked as not stale anymore due to the recent activity.

glours commented 2 years ago

As mentioned by @dcorking this issue sounds to be resolved. By the way the service.mem_limit has been deprecated and should be replace by the service.deploy.limits.memory

@dhruvkb I close the issue, feel free to ping me if you're still able to reproduce it

benmercerdev commented 1 year ago

I solved this by changing '2gb' to '2000000000'