Unable to set rootful mode for applehv mode

[cdrage]

Issue Description

▶ podman version
Client:       Podman Engine
Version:      4.8.3
API Version:  4.8.3
Go Version:   go1.21.5
Git Commit:   85dc30df56566a654700722a4dd190e1b9680ee7
Built:        Wed Jan  3 06:18:44 2024
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      4.7.2
API Version:  4.7.2
Go Version:   go1.21.1
Built:        Tue Oct 31 10:30:33 2023
OS/Arch:      linux/arm64

I'm unable to set --rootful mode for a podman machine running on applehv.

podman machine start --log-level=debug
INFO[0000] podman filtering at log level debug
DEBU[0000] Using Podman machine with `applehv` virtualization provider
DEBU[0000] connection refused: http://localhost:8081/vm/state
Starting machine "podman-machine-default"
DEBU[0000] connection refused: http://localhost:8081/vm/state
DEBU[0000] gvproxy binary being used: /opt/homebrew/Cellar/podman/4.8.3/libexec/podman/gvproxy
DEBU[0000] [-debug -mtu 1500 -ssh-port 58522 -listen-vfkit unixgram:///var/folders/cn/pm5mgrcs5h339nphypxtvd080000gn/T/podman/gvproxy.sock -forward-identity /Users/cdrage/.ssh/podman-machine-default -forward-sock /Users/cdrage/.local/share/containers/podman/machine/applehv/podman.sock -forward-dest /run/user/501/podman/podman.sock -forward-user core -pid-file /var/folders/cn/pm5mgrcs5h339nphypxtvd080000gn/T/podman/gvproxy.pid]
DEBU[0000] gvproxy unixgram socket "/var/folders/cn/pm5mgrcs5h339nphypxtvd080000gn/T/podman/gvproxy.sock" not found: stat /var/folders/cn/pm5mgrcs5h339nphypxtvd080000gn/T/podman/gvproxy.sock: no such file or directory
DEBU[0000] vfkit path is: /usr/local/bin/vfkit
DEBU[0000] first boot detected
DEBU[0000] serving ignition file over /Users/cdrage/.local/share/containers/podman/machine/applehv/ignition.sock
DEBU[0000] listening for ready on: /var/folders/cn/pm5mgrcs5h339nphypxtvd080000gn/T/podman/podman-machine-default_ready.sock
DEBU[0000] waiting for ready notification
DEBU[0000] reading ignition file: /Users/cdrage/.config/containers/podman/machine/applehv/podman-machine-default.ign
DEBU[0000] ignition socket device: /Users/cdrage/.local/share/containers/podman/machine/applehv/ignition.sock
DEBU[0007] ready notification received
WARN[0007] API socket failed ping test

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command:

        podman machine set --rootful

API forwarding listening on:

Another process was listening on the default Docker API socket address.
You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

        export DOCKER_HOST='unix://'

Machine "podman-machine-default" started successfully
DEBU[0007] Called machine start.PersistentPostRunE(podman machine start --log-level=debug)
DEBU[0007] Shutting down engines

Steps to reproduce the issue

Steps to reproduce the issue

1. export CONTAINERS_MACHINE_PROVIDER=applehv
2. podman machine init
3. podman machine set --rootful
4. podman machine start

After podman machine start, you'll see that it's not in rootful mode.

Describe the results you received

See above, machine still stuck in rootless mode.

Describe the results you expected

Rootful mode

podman info output

podman info
host:
  arch: arm64
  buildahVersion: 1.32.0
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.8-2.fc39.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: '
  cpuUtilization:
    idlePercent: 99.34
    systemPercent: 0.39
    userPercent: 0.28
  cpus: 5
  databaseBackend: boltdb
  distribution:
    distribution: fedora
    variant: coreos
    version: "39"
  eventLogger: journald
  freeLocks: 2048
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 6.6.3-200.fc39.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 1618345984
  memTotal: 1982873600
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.8.0-1.fc39.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.8.0
    package: netavark-1.8.0-2.fc39.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.8.0
  ociRuntime:
    name: crun
    package: crun-1.12-1.fc39.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.12
      commit: ce429cb2e277d001c2179df1ac66a470f00802ae
      rundir: /run/user/501/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20231119.g4f1709d-1.fc39.aarch64
    version: |
      pasta 0^20231119.g4f1709d-1.fc39.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.fc39.aarch64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 5m 2.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphRootAllocated: 106769133568
  graphRootUsed: 2611068928
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/user/501/containers
  transientStore: false
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.7.2
  Built: 1698762633
  BuiltTime: Tue Oct 31 10:30:33 2023
  GitCommit: ""
  GoVersion: go1.21.1
  Os: linux
  OsArch: linux/arm64
  Version: 4.7.2

### Podman in a container

No

### Privileged Or Rootless

None

### Upstream Latest Release

Yes

### Additional environment details

Additional environment details

### Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[cdrage]

When doing the set command with debug, it seems to have gone through, but no changes:

▶ podman machine set --rootful=true --log-level=debug
INFO[0000] podman filtering at log level debug
DEBU[0000] Using Podman machine with `applehv` virtualization provider
DEBU[0000] connection refused: http://localhost:8081/vm/state
DEBU[0000] Called machine set.PersistentPostRunE(podman machine set --rootful=true --log-level=debug)
DEBU[0000] Shutting down engines

[jakecorrenti]

/assign