Low ulimits when building with docker-compose/podman-compose

[jRilkef]

Issue Description

When an image is build with docker-compose or podman-compose, the nofile ulimit is 1024 which is too low.

Steps to reproduce the issue

Build the following with either podman-compose or docker-compose that's using podman socket:

# Containerfile
FROM docker.io/library/alpine:3
RUN cat /proc/self/limits

# docker-compose.yml
version: "3.8"
services:
  test:
    image: localhost/test:1
    build:
      dockerfile: Containerfile
      context: .

Describe the results you received

...
STEP 2/2: RUN cat /proc/self/limits
Limit                     Soft Limit           Hard Limit           Units     
Max open files            1024                 1024                 files     
...

Describe the results you expected

I expect this limit to be configurable. Podman has the following overrides set up:

# /etc/security/limits.d/100-podman.conf
...
root soft nofile 131072
root hard nofile 524288
* soft nofile 131072
* hard nofile 524288

# /etc/containers/containers.conf AND /usr/share/containers/containers.conf 
[containers]
default_ulimits = ["nofile=131072:524288",]
...

However, builds using *-compose are not honoring those overrides.

podman info output

$ podman info

```yaml host: arch: amd64 buildahVersion: 1.33.2 cgroupControllers: - cpu - io - memory - pids cgroupManager: systemd cgroupVersion: v2 conmon: package: conmon-2.1.8-2.fc39.x86_64 path: /usr/bin/conmon version: 'conmon version 2.1.8, commit: ' cpuUtilization: idlePercent: 98.78 systemPercent: 0.87 userPercent: 0.35 cpus: 4 databaseBackend: sqlite distribution: distribution: fedora variant: coreos version: "39" eventLogger: journald freeLocks: 2048 hostname: localhost.localdomain idMappings: gidmap: - container_id: 0 host_id: 1000 size: 1 - container_id: 1 host_id: 100000 size: 1000000 uidmap: - container_id: 0 host_id: 501 size: 1 - container_id: 1 host_id: 100000 size: 1000000 kernel: 6.6.8-200.fc39.x86_64 linkmode: dynamic logDriver: journald memFree: 627404800 memTotal: 2059902976 networkBackend: netavark networkBackendInfo: backend: netavark dns: package: aardvark-dns-1.9.0-1.fc39.x86_64 path: /usr/libexec/podman/aardvark-dns version: aardvark-dns 1.9.0 package: netavark-1.9.0-1.fc39.x86_64 path: /usr/libexec/podman/netavark version: netavark 1.9.0 ociRuntime: name: crun package: crun-1.12-1.fc39.x86_64 path: /usr/bin/crun version: |- crun version 1.12 commit: ce429cb2e277d001c2179df1ac66a470f00802ae rundir: /run/user/501/crun spec: 1.0.0 +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL os: linux pasta: executable: /usr/bin/pasta package: passt-0^20231204.gb86afe3-1.fc39.x86_64 version: | pasta 0^20231204.gb86afe3-1.fc39.x86_64 Copyright Red Hat GNU General Public License, version 2 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. remoteSocket: exists: true path: /run/user/501/podman/podman.sock security: apparmorEnabled: false capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT rootless: true seccompEnabled: true seccompProfilePath: /usr/share/containers/seccomp.json selinuxEnabled: true serviceIsRemote: true slirp4netns: executable: /usr/bin/slirp4netns package: slirp4netns-1.2.2-1.fc39.x86_64 version: |- slirp4netns version 1.2.2 commit: 0ee2d87523e906518d34a6b423271e4826f71faf libslirp: 4.7.0 SLIRP_CONFIG_VERSION_MAX: 4 libseccomp: 2.5.3 swapFree: 0 swapTotal: 0 uptime: 0h 18m 19.00s variant: "" plugins: authorization: null log: - k8s-file - none - passthrough - journald network: - bridge - macvlan - ipvlan volume: - local registries: search: - docker.io store: configFile: /var/home/core/.config/containers/storage.conf containerStore: number: 0 paused: 0 running: 0 stopped: 0 graphDriverName: overlay graphOptions: {} graphRoot: /var/home/core/.local/share/containers/storage graphRootAllocated: 106769133568 graphRootUsed: 3016437760 graphStatus: Backing Filesystem: xfs Native Overlay Diff: "true" Supports d_type: "true" Supports shifting: "false" Supports volatile: "true" Using metacopy: "false" imageCopyTmpDir: /var/tmp imageStore: number: 2 runRoot: /run/user/501/containers transientStore: false volumePath: /var/home/core/.local/share/containers/storage/volumes version: APIVersion: 4.8.2 Built: 1702300984 BuiltTime: Mon Dec 11 15:23:04 2023 GitCommit: "" GoVersion: go1.21.4 Os: linux OsArch: linux/amd64 Version: 4.8.2 ```

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

I'm facing the same issue with podman on macOS and with privileged podman in a container.

Additional information

n/a

[rhatdan]

$ ulimit -n 1024

Podman and no other process in rootless mode can override the limits set for the user running the podman command.

If you raise the limits for the user, then the ulimit should increase.

[jRilkef]

During build time, both soft and hard limits seem to inherit from the user's soft limit. During run time, however (aka podman run --entrypoint cat alpine /proc/self/limits), both inherit from the user's hard limit. Is that expected behavior?

[mheon]

That sounds like a bug - Buildah grabbing different limits from Podman. Question would be which to standardize on; I would vote for hard limit as that's likely higher, and we see a lot of folks running out of resources.

[Miaourt]

In the meantime that the fix get downstreamed on every podman installs, is there a solution to manually up the limit beside running as root ?