Enhance Docker for Mac to accommodate Catalina /rootPath (/etc/synthetic.conf) entries

[bodhi-one]

4341

4325

4314

Were all closed without resolution, is that correct?

If this was resolved can you please provide an information update with which version this was fixed? If this is not resolved can someone please schedule a fix?

From #4325

Expected behavior

When I add a real path to Docker as a shared volume, and then reference that path via some path with a symlink, I would expect Docker to resolve a realpath from the desired shared volume path, and compare that with the shared volume paths, rather than the symlink path.

See Steps to reproduce the behavior for an example.

Actual behavior

Docker seems to treat these paths as literal strings. As in, unless the explicit symlink path is added to Docker Prefs, it forbids the directory as a shared volume when it should notice it is allowed (and consequently allow it).

Information

Is it reproducible? Certainly, yes. Is the problem new? Yes. Did the problem appear with an update? All problems appear in updates.

macOS Version: 10.15.3 (Catalina) Diagnostic logs Docker for Mac: 2.2.0.3 (42716) Engine: 19.03.5 Notary: 0.6.1 Compose: 1.25.4 Kubernetes: v1.15.5 Credential Helper: 0.6.3

Steps to reproduce the behavior Install Docker for Mac latest Execute the following to create our "real" path: cd ~/Desktop && mkdir -p docker-sample/hello Execute the following to setup a symlink from "real" to somewhere else: cd docker-sample && ln -s pwd/hello pwd/yo Add the "real" path to Docker for Mac as an allowed shared volume path (this is the only option because of #4318, i.e. one cannot add the symlinked path unless it works fine in the OS dialog, which only a subset of fs links do) Try to run a container using the symbolic path: docker run --rm -v pwd/yo:/data -it ubuntu:latest bash

Manual work around from #4341

federico-piazza commented on Mar 5 I could bypass the UI by editing this file:

~/Library/Group Containers/group.com.docker/settings.json Then added my synthetic link there and fortunately that made the trick:

{ "filesharingDirectories" : [ "\/Users", "\/Volumes", "\/datadrive", "\/private", "\/tmp" ],

ALSO From #4341 Details on when the Docker for Mac UI changed and this became an issue

This behavior has appeared after the new UI release. I have found a post in SO where the guy mentions:

Update I've downgraded progressively up to Docker Community Edition 2.0.0.3 2019-02-15. That seems to be the last version with the old user interface. With this version the folder browser dialog from file sharing displays all the folders and also manual editing of the file paths works. On versions Docker Desktop Community 2.1.0.1 and Docker Desktop Community 2.1.0.2, which have the new UI, it doesn't work.

[bodhi-one]

/lifecycle frozen

[stephen-turner]

Noted, but unfortunately low priority as you are only the fourth user to report it out of our over one million users.

[banagale]

Just to add my voice to this. Working around this took enough time that I recorded my own issue as @stephen-turner linked above: https://github.com/docker/for-mac/issues/5127

At the least, it would be very helpful if the Docker docs for mac specifically addressed the use of synthetic.conf and the workaround necessary to use the system drive path symlinks. Even a link to this issue would would have saved me a bunch of time.

It is also worth noting that compose paths on macos have to point at the actual path, so there is some amount of local dev tooling that should be taken into consideration for Docker on mac.

[bodhi-one]

Update: It appears with Docker for Mac Desktop 3.x (and Big Sur OSX) that NO entries for root paths placed in /etc/synthetic.conf are successfully mapped any longer at all.

Given /apath and this defined in /etc/synthetic.conf pointing to local user ~/apath

Given a docker run command trying to use -v /apath:/containerpath

Using the /path (/apath) you will have no files in the container at /containerpath.

A work around is that you have to use a non-root / path (something which can be seen without going through /etc/synthetic.conf) in order to see your files in the container at /containerpath.

From the above example we have to now use -v ~/apath:/containerpath to see the files in the container.

This is an issue for us because we do use /paths on our Linux servers and our Mac Desktops are testing some server functionality during LCL builds and other regression testing on LCL Mac.

So behavior has now degraded to the point where root entries placed in /etc/synthetic.conf no longer work. Not 100% certain if this is a Big Sur bug, or, perhaps more likely, something that Docker for Mac desktop 3.x needs to add to be aware of some nuance that has changed in Big Sur. Either way, fix should be coded with this combination to be sure it will work.

[banagale]

Update: It appears with Docker for Mac Desktop 3.x (and Big Sur OSX) that NO entries for root paths placed in /etc/synthetic.conf are successfully mapped any longer at all.

I have a /[dirname]/ mapped in Big Sur to a non-system disk path and use it as a mounted volume in a docker-compose and it is working. (I did add it using macos Docker preferences, resources file sharing)

However, when I went to add a new one /home/[dirnamex] it didn't seem to work, I wondered if it was because /home is also a reserved system path by Big Sur.

That was when I realized for local development I just needed to do as you described, @talj-us-ibm-com , which is to use

    volumes:
      - static_volume:/usr/src/app/static
      - /Users/[username]/code/.../[dirname]/:/[dirname]/

rather than

    volumes:
      - static_volume:/usr/src/app/static
      - /[dirname]/:/[dirname]/

I would probably just have local-dev versions that do this going forward. And also just not design anything new that used a directory off the root if at all possible.

[docker-robott]

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[bodhi-one]

/remove-lifecycle

[stephen-turner]

/remove-lifecycle stale

[docker-robott]

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[bodhi-one]

/remove-lifecycle

[stephen-turner]

/remove-lifecycle stale

[docker-robott]

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[banagale]

/remove-lifecycle stale

[docker-robott]

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[bodhi-one]

/lifecycle frozen