podman build provides only nofiles=1024

MartinEmrich commented 8 months ago

Issue Description

The maximum number of open files does not match between podman run and podman build

I am running "rootless", Podman 4.3.1.

On my System, I get max open files of 1M (Debian 12 on WSL2, default settings):

$ cat /proc/self/limits | grep "Max open files"
Max open files            1024                 1048576              files

With podman run, I also get enough:

$ podman run --rm -it debian:bookworm cat /proc/self/limits | grep "Max open files"
Max open files            1048576              1048576              files

But with podman build, I only geht 1024, and I see no reason why:

echo -e "FROM debian:bookworm\nRUN cat /proc/self/limits" | podman build --no-cache -f /dev/stdin | grep "Max open files"
Max open files            1024                 1024                 files

The --ulimit command line option is not a solution. I have a symlink from docker in my path and run build scripts from third parties, which I cannot modify easily.

Steps to reproduce the issue

Steps to reproduce the issue:

Oneliner to visualize:

echo -e "FROM debian:bookworm\nRUN cat /proc/self/limits" | podman build --no-cache -f /dev/stdin | grep "Max open files"
Max open files            1024                 1024                 files

Describe the results you received

Severely limited nofile ulimit value (and possibly others?)

Describe the results you expected

Exactly the ulimits my calling shell currently has, or at least the same behaviour as podman run

podman info output

host:
  arch: amd64
  buildahVersion: 1.28.2
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon_2.1.6+ds1-1_amd64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: unknown'
  cpuUtilization:
    idlePercent: 99.54
    systemPercent: 0.28
    userPercent: 0.18
  cpus: 16
  distribution:
    codename: bookworm
    distribution: debian
    version: "12"
  eventLogger: file
  hostname: nb-494
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.15.146.1-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 16184242176
  memTotal: 20976140288
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun_1.8.1-1+deb12u1_amd64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.1
      commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30
      rundir: /mnt/wslg/runtime-dir/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /mnt/wslg/runtime-dir/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns_1.2.0-1_amd64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.4
  swapFree: 0
  swapTotal: 0
  uptime: 8h 30m 58.00s (Approximately 0.33 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
   XXX
store:
  configFile: /home/XXX/.config/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 0
    stopped: 2
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/XXX/.local/share/containers/storage
  graphRootAllocated: 269427478528
  graphRootUsed: 160556539904
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 456
  runRoot: /mnt/wslg/runtime-dir/containers
  volumePath: /home/XXX/.local/share/containers/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.19.8
  Os: linux
  OsArch: linux/amd64
  Version: 4.3.1

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

No

Additional environment details

No response

Additional information

setting the soft limit ulimit -n 1048576 has no effect.
setting the ulimit in ~/.config/containers/containers.conf also has no effect.

rhatdan commented 8 months ago

This is fixed in podman 5.0

MartinEmrich commented 8 months ago

@rhatdan thanks!

For anyone finding this: Until that is released, this wrapper (earlier in the $PATH) works ok for me:

#!/bin/bash

REAL_PODMAN=/usr/bin/podman

if [[ "$*" =~ "build " ]]; then
  shift
  exec ${REAL_PODMAN} build --ulimit=nofile=1048576:1048576 "$@"
fi

exec ${REAL_PODMAN} "$@"

containers / podman