`podman kube play` not respecting `io.podman.annotations.userns` annotation

Issue Description

I was using podman kube play --replace --no-hosts --userns=keep-id --start=false ./kube.yaml to start multiple containers, I realized only one of them actually supported rootless operation, so I tried specifying io.podman.annotations.userns on only the ones that did, but they immediate show no permissions and fail when I start them up.

Steps to reproduce the issue

Run a rootless container with --userns=keep-id flag with podman kube play
Then try if that works, then transfer it into the kube.yaml file's io.podman.annotations.userns and run without the --userns=keep-id flag
The rootless container should no longer work

Describe the results you received

When trying to run the gitea-rootless container I get the following:

chmod: /var/lib/gitea/git: Operation not permitted
/var/lib/gitea/git is not writable
docker setup failed
chmod: /var/lib/gitea/git: Operation not permitted
/var/lib/gitea/git is not writable
docker setup failed

Describe the results you expected

For keep-id to be applied to just the pod definitions it appears on

podman info output

host:
  arch: amd64
  buildahVersion: 1.31.2
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon_2:2.1.12-0ubuntu22.04+obs20.6_amd64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 93.93
    systemPercent: 1.05
    userPercent: 5.02
  cpus: 6
  databaseBackend: boltdb
  distribution:
    codename: jammy
    distribution: ubuntu
    version: "22.04"
  eventLogger: journald
  freeLocks: 2042
  hostname: dev-server
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 6.5.0-35-generic
  linkmode: dynamic
  logDriver: journald
  memFree: 3956187136
  memTotal: 33506168832
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns_1.6.0-0ubuntu22.04+obs34.25_amd64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.6.0
    package: netavark_1.3.0-0ubuntu22.04+obs22.9_amd64
    path: /usr/libexec/podman/netavark
    version: netavark 1.3.0
  ociRuntime:
    name: crun
    package: crun_101:1.14.4-0ubuntu22.04+obs70.24_amd64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns_1.3.1-0ubuntu22.04+obs15.8_amd64
    version: |-
      slirp4netns version 1.3.1
      commit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 83h 58m 54.00s (Approximately 3.46 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - ghcr.io
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/josh/.config/containers/storage.conf
  containerStore:
    number: 4
    paused: 0
    running: 0
    stopped: 4
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/josh/.local/share/containers/storage
  graphRootAllocated: 461755645952
  graphRootUsed: 65656848384
  graphStatus:
    Backing Filesystem: zfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 9
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/josh/.local/share/containers/storage/volumes
version:
  APIVersion: 4.6.2
  Built: 0
  BuiltTime: Wed Dec 31 16:00:00 1969
  GitCommit: ""
  GoVersion: go1.18.1
  Os: linux
  OsArch: linux/amd64
  Version: 4.6.2

Podman in a container

Privileged Or Rootless

None

Upstream Latest Release

Additional environment details

No response

Additional information

4.6.2 is the latest I can run since that's the most recent one that builds for Ubuntu22.04

Also, just realized after writing my scripts to explicitly run each pod yaml separately, with the flag instead, all my kube.yaml are pod definitions, not deployments, if that effects it.

containers / podman