NerdCTL Fails to Find Bind Mount Path in /private/tmp on macOS Catalina

[Alexhuszagh]

Description

Version Info

The host OS is macOS, version 10.15.7, and the Lima version is 0.21.

Bug Description

On macOS, /tmp is a link to /private/tmp, and code that canonicalizes paths will use /private/tmp rather than /tmp for the host directory. A quick step to reproduce this is:

# to install grealpath
$ brew install coreutils
$ brew install lima
$ cd /tmp/lima
$ pwd
/tmp/lima
$ grealpath .
/private/tmp/lima
$ limactl start
$ cwd=$(grealpath .)
$ nerdctl.lima run --rm -it -v "${cwd}":/"${cwd}" ubuntu:20.04 bash
FATA[0000] no such file or directory
$ cwd=$(pwd)
$ nerdctl.lima run --rm -it -v "${cwd}":/"${cwd}" ubuntu:20.04 bash
# this works

My Lima info is the following:

$ nerdctl.lima info
Client:
 Namespace: default
 Debug Mode:    false

Server:
 Server Version: v1.6.6
 Storage Driver: overlayfs
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Log: fluentd journald json-file
  Storage: native overlayfs stargz fuse-overlayfs
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
  rootless
 Kernel Version: 5.15.0-39-generic
 Operating System: Ubuntu 22.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.827GiB
 Name: lima-default
 ID: 73a6444c-8e28-4908-a1fa-0b1b74a3f7f2

WARNING: AppArmor profile "nerdctl-default" is not loaded.
         Use 'sudo nerdctl apparmor load' if you prefer to use AppArmor with rootless mode.
         This warning is negligible if you do not intend to use AppArmor.
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

I am using all the default configurations.

Expected Behavior

Lima should properly find and then mount the /private/tmp directory, similar to Docker. This is also an issue in podman. Since it's a sticky directory, and therefore should always be present and a link to the same path, it may be possible to just remove the /private prefix if the canonicalized path starts with /private/tmp.

[jandubois]

This is working as expected; the default config maps /tmp and not /private/tmp into the VM: https://github.com/lima-vm/lima/blob/de9f354a24c1aa4abeca021af796999ec0019cb7/examples/default.yaml#L85-L88

If you want to references files via /private/tmp/lima then you have to update the mount definition in lima.yaml.

[Alexhuszagh]

I guess then I've got 2 follow-up questions:

1. Why aren't symlinks followed if they resolved to the mounted directory, especially for cases where they're not meant to be transparent to the user (such as /private/tmp, which is meant to be compatible with UNIX-like systems)?
2. Can there be an option to do so? IE, canonicalize the mount directories first? If it's opt-in, then it would be quite nice. Allowing users to follow symlinks (opt-in though) in mounted directories outside feels like (IMO) quite a larger surface area than this.

Something like:

 - location: "/tmp/lima" 
   # 🟢 Builtin default: false 
   # 🔵 This file: true (only for "/tmp/lima") 
   writable: true 
   # Resolve any symlinks prior to mounting.
   # That is, if /tmp/a goes to /tmp/b, allow
   # either /tmp/a or /tmp/b to be mounted.
   canonicalize: true 

[jandubois]

1. Why aren't symlinks followed if they resolved to the mounted directory, especially for cases where they're not meant to be transparent to the user (such as /private/tmp, which is meant to be compatible with UNIX-like systems)?

I don't follow, the whole point of having a symlink is to be "transparent" to the user, so you can use /tmp and not worry about where the data is actually stored.

2. Can there be an option to do so? IE, canonicalize the mount directories first? If it's opt-in, then it would be quite nice. Allowing users to follow symlinks (opt-in though) in mounted directories outside feels like (IMO) quite a larger surface area than this.

Theoretically yes, but I don't see the point; why not put the target of the symlink into the lima.yaml file, if that's what you want:

mounts:
- location: /private/tmp/lima
  writable: true

This is a lot more explicit about what you want to happen.