Error creating build container in /var/cache/containers

Issue Description

I am running podman-compose as follows which in turn calls podman:

$ /usr/bin/unshare -r /usr/bin/podman-compose --podman-args '--root /tmp/containers/root --runroot /tmp/containers/run --tmpdir /tmp/containers/libpod --storage-driver vfs --transient-store' -f compose.yaml build
podman-compose version: 1.0.6
['podman', '--version', '']
using podman version: 4.9.4
podman build --root /tmp/containers/root --runroot /tmp/containers/run --tmpdir /tmp/containers/libpod --storage-driver vfs --transient-store -f ./Dockerfile -t alpine:latest .
STEP 1/4: FROM alpine:latest
Error: creating build container: mkdir /var/cache/containers: permission denied
exit code: 125

The reason why I am using unshare -r is because the command is ultimately run inside of an environment where further uid delegation is not possible. But that is a different story.

However, when creating the build container podman attempts to create it in /var/cache/containers which is not accessible to podman because of the root uid mapping. I also don't want podman to write anything there but everything has to happen inside a sandbox. As it can be seen from the command above root, runroot, and tmpdir are set. I would expect everything to be placed inside these directories but yet podman still attempts to access /var/cache.

Is there another option that can be set (and which I missed) or is that indeed a bug?

Steps to reproduce the issue

Just run:

$ unshare -r podman build --root /tmp/containers/root --runroot /tmp/containers/run --tmpdir /tmp/containers/libpod --storage-driver vfs --transient-store -f ./Dockerfile -t alpine:latest .
STEP 1/4: FROM alpine:latest
Error: creating build container: mkdir /var/cache/containers: permission denied

Describe the results you received

The following error message:

STEP 1/4: FROM alpine:latest
Error: creating build container: mkdir /var/cache/containers: permission denied

Describe the results you expected

This is the output of a successful build:

STEP 1/4: FROM alpine:latest
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob 4abcf2066143 done   | 
Copying config 05455a0888 done   | 
Writing manifest to image destination
STEP 2/4: RUN apk add p11-kit-server
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/4) Installing libffi (3.4.4-r3)
(2/4) Installing libtasn1 (4.19.0-r2)
(3/4) Installing p11-kit (0.25.3-r0)
(4/4) Installing p11-kit-server (0.25.3-r0)
Executing busybox-1.36.1-r15.trigger
OK: 10 MiB in 19 packages
--> b15600d605df
STEP 3/4: RUN apk add gnutls-utils
(1/6) Installing gmp (6.3.0-r0)
(2/6) Installing nettle (3.9.1-r0)
(3/6) Installing libunistring (1.1-r2)
(4/6) Installing libidn2 (2.3.4-r4)
(5/6) Installing gnutls (3.8.4-r0)
(6/6) Installing gnutls-utils (3.8.4-r0)
Executing busybox-1.36.1-r15.trigger
OK: 15 MiB in 25 packages
--> b84b65ea5628
STEP 4/4: CMD tail -f /dev/null
COMMIT alpine:latest
--> 9aa6c607cf3c
Successfully tagged localhost/alpine:latest
9aa6c607cf3c26d036813358e20605089545486865c9fcb08f6f8054967ba72d

podman info output

$ unshare -r podman info --root /tmp/containers/root --runroot /tmp/containers/run --tmpdir /tmp/containers/libpod --storage-driver vfs --transient-store
host:
  arch: amd64
  buildahVersion: 1.33.7
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc38.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 96.96
    systemPercent: 1.92
    userPercent: 1.12
  cpus: 128
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: workstation
    version: "38"
  eventLogger: journald
  freeLocks: 2048
  hostname: threaddy
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.7.9-100.fc38.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 15228346368
  memTotal: 134918381568
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-1.fc38.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-1.fc38.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.fc38.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240220.g1e6f92b-1.fc38.x86_64
    version: |
      pasta 0^20240220.g1e6f92b-1.fc38.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.fc38.x86_64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 12854718464
  swapTotal: 12884893696
  uptime: 263h 13m 43.00s (Approximately 10.96 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: vfs
  graphOptions: {}
  graphRoot: /tmp/containers/root
  graphRootAllocated: 67459190784
  graphRootUsed: 57450496
  graphStatus: {}
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 4
  runRoot: /tmp/containers/run
  transientStore: true
  volumePath: /tmp/containers/root/volumes
version:
  APIVersion: 4.9.4
  Built: 1711446116
  BuiltTime: Tue Mar 26 02:41:56 2024
  GitCommit: ""
  GoVersion: go1.21.8
  Os: linux
  OsArch: linux/amd64
  Version: 4.9.4

Podman in a container

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

containers / podman