Bug: Setting `--root` ignores `additionalimagestores` in `storage.conf`

gabyx commented 8 months ago

Issue Description

Podman: 4.3.9

Run

podman --privileged -v "$HOME/.local/share/containers/storage:/var/lib/shared" --rm -it quay.io/podman:4.3.9 podman --root /whatever images Does not show any images.
podman --privileged -v "$HOME/.local/share/containers/storage:/var/lib/shared" --rm -it quay.io/podman:4.3.9 podman images Should show all images you have on your system

Its strange that the additionalimagestores in /etc/containers/storage.confg gets ignored when setting --root? Is that due to graphOptions beeing ignored?

https://github.com/containers/podman/blob/main/pkg/domain/infra/runtime_libpod.go#L86

Steps to reproduce the issue

podman --privileged -v "$HOME/.local/share/containers/storage:/var/lib/shared" --rm -it quay.io/podman:4.3.9 podman --root /whatever images
Output should show images on your local machine but its empty!

Describe the results you received

No images found.

Describe the results you expected

Images found in /var/lib/shared which is mounted.

podman info output

host:
  arch: amd64
  buildahVersion: 1.33.5
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /nix/store/wp1wic4bsd28qb7hs04dr7dsgls5j8pf-conmon-2.1.10/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 97.93
    systemPercent: 0.88
    userPercent: 1.19
  cpus: 32
  databaseBackend: sqlite
  distribution:
    codename: uakari
    distribution: nixos
    version: "24.05"
  eventLogger: journald
  freeLocks: 2035
  hostname: linux-nixos
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 100
      size: 1
    - container_id: 1
      host_id: 100000
      size: 10000000
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 10000000
  kernel: 6.6.19
  linkmode: dynamic
  logDriver: journald
  memFree: 26401583104
  memTotal: 67342225408
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: Unknown
      path: /nix/store/vwrqp3pfj17p0zd5gdnrkrvr702z838r-podman-4.9.3/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: Unknown
    path: /nix/store/vwrqp3pfj17p0zd5gdnrkrvr702z838r-podman-4.9.3/libexec/podman/netavark
    version: netavark 1.7.0
  ociRuntime:
    name: crun
    package: Unknown
    path: /nix/store/h9wpwj1zjz6qjw3q331k757cq27kmj9k-crun-1.14.4/bin/crun
    version: |-
      crun version 1.14.4
      commit: 1.14.4
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    exists: false
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: ""
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /nix/store/vwrqp3pfj17p0zd5gdnrkrvr702z838r-podman-4.9.3/libexec/podman/slirp4netns
    package: Unknown
    version: |-
      slirp4netns version 1.2.3
      commit: c22fde291bb35b354e6ca44d13be181c76a0a432
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.5
  swapFree: 77309407232
  swapTotal: 77309407232
  uptime: 8h 58m 35.00s (Approximately 0.33 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /home/nixos/.config/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 2
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/nixos/.local/share/containers/storage
  graphRootAllocated: 1049659179008
  graphRootUsed: 216383045632
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 90
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/nixos/.local/share/containers/storage/volumes
version:
  APIVersion: 4.9.3
  Built: 315532800
  BuiltTime: Tue Jan  1 01:00:00 1980
  GitCommit: ""
  GoVersion: go1.21.7
  Os: linux
  OsArch: linux/amd64
  Version: 4.9.3

Podman in a container

Yes

Privileged Or Rootless

Privileged

Upstream Latest Release

Yes

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

giuseppe commented 8 months ago

that is by design, when you change the --root to use, we cannot assume the additional stores are compatible with the graph driver currently in use. You need to pick the options you'd like to use with --storage-opt.

gabyx commented 8 months ago

@giuseppe :

Thanks I just realized that, is there an option for podman to use a file as storage-opts basically saying use the same file again --storage-opts-file /etc/containers/storage.conf? I just wanted to change the root to have some isolation in nesting containers:

https://github.com/containers/podman/discussions/22049#discussioncomment-8827386

giuseppe commented 8 months ago

you can use the CONTAINERS_STORAGE_CONF environment variable to point to a different storage.conf file

gabyx commented 8 months ago

@giuseppe : I guess that this env. variable does not work (on my machine) with

 CONTAINERS_STORAGE_CONF=/etc/containers/storage.conf \
    podman  --root /podman-root/root "$@"

because I guess --root has precedence and although CONTAINERS_... has been used its set to empty in the code because of --root.

gabyx commented 8 months ago

But I can of course just make another storage.conf file and use that instead of doing --root etc.

containers / podman