Permission Denied Error when Using podman generate kube with Bind Mount Options

[realSConway]

Issue Description

When using podman generate kube command to create Kubernetes YAML files for a pod containing two containers, I encountered an issue with bind-mounting directories. Specifically, when I set the annotations: bind-mount-options for the database directory, the configuration file becomes unreadable, leading to the following error message: Could not open "/etc/tor/torrc": Permission denied Setting annotations: bind-mount-options for the conf file results in another error related to the database directory: Exception in main! boost::filesystem::status: Permission denied [system:13]: "/home/database/lmdb

I have moved both item (hidden_service and torrc) to /data/monero but error still occurs.

Steps to reproduce the issue

1. Create pod:

   podman network create net-monero
   podman pod create --name=monero --network=net-monero --publish=18080-18089:18080-18089
   podman create --name=tor-xmr --pod=monero --volume=./torrc:/etc/tor/torrc:ro --volume=./hidden_service:/var/lib/tor/hidden_service:Z osminogin/tor-simple
   podman create --name=xmr --pod=monero --volume=/data/monero:/home/monero/.bitmonero:Z sethsimmons/simple-monerod:latest

2. generate kube file, podman generate kube monero

output:

# Save the output of this file and use kubectl create -f to import
# it into Kubernetes.
#
# Created with podman-4.5.1
  annotations:
    #bind-mount-options: /home/support/podman/documents/webtest/hidden_service:Z
    bind-mount-options: /data/monero:Z
  creationTimestamp: "2023-07-28T15:12:36Z"
  labels:
    app: monero
  name: monero
  restartPolicy: Never
  containers:
  - image: docker.io/osminogin/tor-simple:latest
    name: tor-xmr-test
    securityContext: {}
    volumeMounts:
    - name: home-support-podman-documents-webtest-torrc-host-0
      mountPath: /etc/tor/torrc
    - name: home-support-podman-documents-webtest-hidden_service-host-1
      mountPath: /var/lib/tor/hidden_service
  - image: docker.io/sethsimmons/simple-monerod:latest
    name: xmr
    securityContext: {}
    volumeMounts:
    - name: data-monero-host-0
      mountPath: /home/monero/.bitmonero
  volumes:
  - name: home-support-podman-documents-webtest-torrc-host-0
    hostPath:
      path: /home/support/podman/documents/webtest/torrc
      type: File
  - name: home-support-podman-documents-webtest-hidden_service-host-1
    hostPath:
      path: /home/support/podman/documents/webtest/hidden_service
      type: Directory
  - name: data-monero-host-0
    hostPath:
      path: /data/monero
      type: Directory

3. Run kube file: podman kube play ./pod-monero.yaml
4. Notice permission errors when swapping commented bind-mount-options

Describe the results you received

Permission denied

Describe the results you expected

The pod should be successfully created with both containers, and the specified directories should be properly mounted. The configuration file should be readable, and no permission denied errors should occur.

podman info output

# podman version
Client:       Podman Engine
Version:      4.6.0
API Version:  4.6.0
Go Version:   go1.20.5
Built:        Fri Jul 21 02:00:00 2023
OS/Arch:      linux/amd64

# podman info
host:
  arch: amd64
  buildahVersion: 1.31.0
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.7-2.1.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.7, commit: unknown'
  cpuUtilization:
    idlePercent: 86.21
    systemPercent: 5.68
    userPercent: 8.1
  cpus: 4
  databaseBackend: boltdb
  distribution:
    distribution: '"opensuse-microos"'
    version: "20230729"
  eventLogger: journald
  freeLocks: 2025
  hostname: srv04
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    - container_id: 65537
      host_id: 165536
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    - container_id: 65537
      host_id: 165536
      size: 65536
  kernel: 6.4.6-1-default
  linkmode: dynamic
  logDriver: journald
  memFree: 196988928
  memTotal: 17583128576
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.7.0-2.1.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.7.0
    package: netavark-1.7.0-2.1.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.7.0
  ociRuntime:
    name: runc
    package: runc-1.1.8-1.1.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.1.8
      commit: v1.1.8-0-g82f18fe0e44a
      spec: 1.0.2-dev
      go: go1.20.5
      libseccomp: 2.5.4
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-1.3.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: unknown
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 5
      libseccomp: 2.5.4
  swapFree: 0
  swapTotal: 0
  uptime: 12h 47m 54.00s (Approximately 0.50 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.opensuse.org
  - registry.suse.com
  - docker.io
store:
  configFile: /home/support/.config/containers/storage.conf
  containerStore:
    number: 12
    paused: 0
    running: 9
    stopped: 3
  graphDriverName: btrfs
  graphOptions: {}
  graphRoot: /var/podman/support/storage
  graphRootAllocated: 698137776128
  graphRootUsed: 5427335168
  graphStatus:
    Build Version: Btrfs v6.3
    Library Version: "102"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 13
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /var/podman/support/storage/volumes
version:
  APIVersion: 4.6.0
  Built: 1689897600
  BuiltTime: Fri Jul 21 02:00:00 2023
  GitCommit: ""
  GoVersion: go1.20.5
  Os: linux
  OsArch: linux/amd64
  Version: 4.6.0

### Podman in a container

No

### Privileged Or Rootless

Rootless

### Upstream Latest Release

Yes

### Additional environment details

Additional environment details

### Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[rhatdan]

SELinux is blocking the access.

[realSConway]

I have read about rootless and SELinux, specifically: https://www.redhat.com/sysadmin/user-namespaces-selinux-rootless-containers

Shouldn't pods be sharing same namespace?

[rhatdan]

Containers within a Pod share the same namespaces and SELinux labels. If you are sharing a volume between two different pods, then I usually advise to label the volumes with s0 level, so that all of the containers within the pod, and all containers in general, can read/write the content in the volume. From an SELinux point of view.

[realSConway]

Ok, then that isn't my issue, volumes are not shared between containers. It seems I can only mount one volume which is defined in annotations > bind-mount-options?

...etc
annotations:
    #bind-mount-options: /home/support/podman/documents/webtest/hidden_service:Z
    bind-mount-options: /data/monero:Z
...etc

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

:z versus :Z would relabel with a shared label. I don't think this is a podman issue, so closing.

[AlexRamallo]

I think this is still an issue. The problem is that with the bind-mount-options entry, it's only possible to specify a single mount. A trivial pod with two volumes (not shared) that depend on the :Z flag won't work because the generated yaml file will only select one of the volumes to write into bind-mount-options.

I ran into this today trying to use the generate kube command for the first time. My pod fails because one of the two volumes doesn't get relabeled automatically.

It seems the only workaround currently is to fix SELinux issues manually, and not expect podman generate/podman play to do it for you.

For extra clarity, here's an error from podman 4.6.2 if I try to edit the generated YAML file to use a list in order to support two mounts:

# Created with podman-4.6.2
apiVersion: v1
kind: Pod
metadata:
  annotations:
    bind-mount-options:
      - /var/data/myservice/app:Z
      - /var/data/myservice/db:Z

#...

podman play kube ... produces this error:

Error: unable to read YAML as Kube Pod: error unmarshaling JSON: while decoding JSON: json: cannot unmarshal array into Go struct field ObjectMeta.metadata.annotations of type string

I assume the solution would involve some refactoring in that general direction, but I'm not familiar enough with this project to work on it myself.