user id not working in container on MacOS

[nsheff]

I have a different problem now:

Levis-Air:~ lw391$ bulker activate waldronlab/levi
Bulker config: /Users/lw391/bulker_config.yaml
Activating bulker crate: waldronlab/levi

bulker-3.2$ cd git/hub.bulker.io/
bulker-3.2$ git pull
No user exists for uid 501
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
bulker-3.2$ _git
Starting interactive docker shell for image 'samueldebruyn/debian-git' and command 'git'
I have no name!@docker-desktop:/Users/lw391/git/hub.bulker.io$ ls .git
COMMIT_EDITMSG  FETCH_HEAD  HEAD  ORIG_HEAD  config  description  hooks  index  info  logs  objects  packed-refs  refs
I have no name!@docker-desktop:/Users/lw391/git/hub.bulker.io$ whoami
whoami: cannot find name for user ID 501
I have no name!@docker-desktop:/Users/lw391/git/hub.bulker.io$ 

Here is my bulker_config.yaml:

bulker:
  volumes: ['/tmp', '$HOME']
  envvars: ['DISPLAY']
  registry_url: http://hub.bulker.io/
  default_crate_folder: ${HOME}/bulker_crates
  singularity_image_folder: ${HOME}/simages
  container_engine: docker
  default_namespace: bulker
  executable_template: templates/docker_executable.jinja2
  shell_template: templates/docker_shell.jinja2
  build_template: templates/docker_build.jinja2
  crates:
    bulker:
      demo:
        default: /Users/lw391/bulker_crates/bulker/demo/default
    bioconductor:
      bioconductor_full:
        default:
          docker_args: --volume=${HOME}/.local/lib/R:/usr/local/lib/R/host-site-library
        devel:
          docker_args: --volume=${HOME}/.local/lib/Rdev:/usr/local/lib/R/host-site-library
      levi:
        default: /Users/lw391/bulker_crates/bulker/levi/default
    waldronlab:
      bioconductor:
        default: /Users/lw391/bulker_crates/waldronlab/bioconductor/default
      levi:
        default: /Users/lw391/bulker_crates/waldronlab/levi/default

Originally posted by @lwaldron in https://github.com/databio/bulker/issues/28#issuecomment-544713067

[nsheff]

Hmm... clearly the user isn't getting correctly passed into the container.

This is how I am doing it: https://github.com/databio/bulker/blob/21e1bf2a4870f382336aebc78bdde831e37c9a17/bulker/templates/docker_executable.jinja2#L6

running git pull works for me with that container:

git pull
Updating 4db47c6..21fa2a6
Fast-forward
 databio/refgenie_0.7.0.yaml  |  2 +-
 waldronlab/bioconductor.yaml | 12 ++++++++++++
 waldronlab/levi.yaml         | 10 ++++++++++
 3 files changed, 23 insertions(+), 1 deletion(-)
 create mode 100644 waldronlab/bioconductor.yaml
 create mode 100644 waldronlab/levi.yaml

Can you post the output of:

cat `which git`

?

[nsheff]

Ok I can explain it I think... where do macs put their user tables? Probably this is specific to linux:

https://github.com/databio/bulker/blob/21e1bf2a4870f382336aebc78bdde831e37c9a17/bulker/templates/docker_executable.jinja2#L10-L13

[nsheff]

Perhaps related to this?

https://stackoverflow.com/questions/40165921/docker-on-the-mac-separates-internal-and-external-file-ownerships-not-so-on-lin

I don't have any experience using docker on MacOS -- how do you normally solve the problem of mapping a local mac user into a docker container?

[nsheff]

@lwaldron based on my reading, on a mac you can actually just take those lines out of the template, because docker isn't running natively on a mac so you're running in some kind of virtualization that already takes care of the user issues...

so they may actually be causing problems. does that make sense?

[nsheff]

btw, @michalstolarczyk has tried this on a mac and it worked.

@michalstolarczyk could you see if you can reproduce the error above?

[stolarczyk]

installed dev version of bulker, used Levi's config. Everything worked as expected:

[mstolarczyk@MichalsMBP bulker](dev): bulker load waldronlab/levi -c ~/levis_cfg.yaml
Bulker config: /Users/mstolarczyk/levis_cfg.yaml
Got URL: http://hub.bulker.io/waldronlab/levi.yaml
Loading manifest: 'waldronlab/levi:default'. Activate with 'bulker activate waldronlab/levi:default'.
Commands available: git, emacs
[mstolarczyk@MichalsMBP bulker](dev): bulker activate waldronlab/levi -c ~/levis_cfg.yaml
Bulker config: /Users/mstolarczyk/levis_cfg.yaml
Activating bulker crate: waldronlab/levi

[mstolarczyk@MichalsMBP bulker]: cd ../hub.bulker.io
[mstolarczyk@MichalsMBP hub.bulker.io]: git pull
Unable to find image 'samueldebruyn/debian-git:latest' locally
latest: Pulling from samueldebruyn/debian-git
2e5698f08c0a: Pull complete 
a7a83470264f: Pull complete 
2c51fef1aa3d: Pull complete 
Digest: sha256:6fda212f1f62ba382143c797d771dee1f4aa505520c277195de5536f2926309f
Status: Downloaded newer image for samueldebruyn/debian-git:latest
Already up-to-date.

[nsheff]

thanks for testing that @michalstolarczyk

[lwaldron]

If I understand correctly, just removing the lines:

--volume="/etc/group:/etc/group:ro" \ 
 --volume="/etc/passwd:/etc/passwd:ro" \ 
 --volume="/etc/shadow:/etc/shadow:ro"  \ 
 --volume="/etc/sudoers.d:/etc/sudoers.d:ro" \ 

from, in my case:

~/Library/Python/2.7/lib/python/site-packages/bulker/templates/docker_executable.jinja2

should solve my user ID problem? I guess there is something else involved, because no matter what I do to that file, then do bulker activate waldronlab/levi, it makes no difference (even if I fill the file with junk). Pardon my ignorance!

[nsheff]

no matter what I do to that file,...

That's just a template file. If you change that file, you'll have to re-load the crate before you'll notice any change. But anyway, that was a speculation that michal proved wrong, because it works as is on his mac. So, it's not what I was thinking. I'm at a loss because the default works on mac and linux here so I don't know why it's not working for you.

Anyway, I need you to post the output of:

cat `which git`

That might give a clue...

[nsheff]

Also -- do you have the same problem with other containers? for example, try loading databio/nsheff or one of the other manifests.

[lwaldron]

Here is the requested output:

Levis-MacBook-Pro:~ lwaldron$ bulker activate waldronlab/levi
Bulker config: /Users/lwaldron/bulker_config.yaml
Activating bulker crate: waldronlab/levi

bulker-3.2$ which git
/Users/lwaldron/bulker_crates/waldronlab/levi/default/git
bulker-3.2$ cat `which git`
#!/bin/sh

docker run --rm --init \
  --user=$(id -u):$(id -g) \
  --network="host" \
  --env "DISPLAY" \
  --volume "/tmp:/tmp" \
  --volume "$HOME:$HOME" \
  --volume="/etc/group:/etc/group:ro" \
  --volume="/etc/passwd:/etc/passwd:ro" \
  --volume="/etc/shadow:/etc/shadow:ro"  \
  --volume="/etc/sudoers.d:/etc/sudoers.d:ro" \
  --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
  --workdir="`pwd`" \
  samueldebruyn/debian-git git "$@"bulker-3.2$ 

I see the same behavior when using git from databio/nsheff, e.g.:

Levis-MacBook-Pro:~ lwaldron$ bulker load databio/nsheff
Bulker config: /Users/lwaldron/bulker_config.yaml
Got URL: http://hub.bulker.io/databio/nsheff.yaml
Loading manifest: 'databio/nsheff:default'. Activate with 'bulker activate databio/nsheff:default'.
Commands available: jekyll, latex, ffmpeg, git, igv, bibtool, gnucash, inkscape, jabref, jabref-cl, libre, liquify, mkdocs, pandoc, R, rr, rd, Rscript, subl, redis-server, redis-cli
Levis-MacBook-Pro:~ lwaldron$ bulker activate databio/nsheff
Bulker config: /Users/lwaldron/bulker_config.yaml
Activating bulker crate: databio/nsheff
bulker-3.2$ _git
Starting interactive docker shell for image 'nsheff/git' and command 'git'
I have no name!@docker-desktop:/Users/lwaldron/dockerhome/git/hub.bulker.io$

But I have made some progress by removing the lines you referenced above from the template and re-loading the crate (same behavior with waldronlab/levi or databio/nsheff). Note the reduced number of lines in the git executable, and the correct preservation of my username lwaldron. Now I'm only prevented from using git only by being asked for my ssh password anew and these these containers not having /usr/bin/ssh-askpass.

bulker-3.2$ cat `which git`
#!/bin/sh

docker run --rm --init \
  --network="host" \
  --env "DISPLAY" \
  --volume "/tmp:/tmp" \
  --volume "$HOME:$HOME" \
  --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
  --workdir="`pwd`" \
  samueldebruyn/debian-git git "$@"bulker-3.2$ 
bulker-3.2$ whoami
lwaldron
bulker-3.2$ git pull
ssh_askpass: exec(/usr/bin/ssh-askpass): No such file or directory
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
bulker-3.2$ 

[nsheff]

Out of curiosity, what version of MacOS are you running?

[nsheff]

Also, is this problem now specific to that images, or for all images?

Is it specific to git images?

[lwaldron]

I'm running Mac OSX Mojave 10.14.6. I got the ssh_askpass error when running git on both waldronlab/levi and databio/nsheff. I realized that my above whoami was not from within a Docker shell; when I do so after removing the 5 user-related lines from the docker_executable.jinja2 and docker_shell.jinja2 templates, I am root user and my container home directory doesn't contain anything from my host home directory:

Levis-MacBook-Pro:hub.bulker.io lwaldron$ bulker load databio/nsheff
Bulker config: /Users/lwaldron/bulker_config.yaml
Got URL: http://hub.bulker.io/databio/nsheff.yaml
That manifest has already been loaded. Overwrite? [y/N] y
Removing all executables in: /Users/lwaldron/bulker_crates/databio/nsheff/default
Loading manifest: 'databio/nsheff:default'. Activate with 'bulker activate databio/nsheff:default'.
Commands available: jekyll, latex, ffmpeg, git, igv, bibtool, gnucash, inkscape, jabref, jabref-cl, libre, liquify, mkdocs, pandoc, R, rr, rd, Rscript, subl, redis-server, redis-cli
Levis-MacBook-Pro:hub.bulker.io lwaldron$ bulker activate databio/nsheff
Bulker config: /Users/lwaldron/bulker_config.yaml
Activating bulker crate: databio/nsheff

bulker-3.2$ cat `which _git`
#!/bin/sh
echo "Starting interactive docker shell for image 'nsheff/git' and command 'git'"
docker run --rm --init -it \
  --network="host" \
  --env "DISPLAY" \
  --volume "/tmp:/tmp" \
  --volume "$HOME:$HOME" \
  --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
  --workdir="`pwd`" \
  nsheff/git bashbulker-3.2$ _git
Starting interactive docker shell for image 'nsheff/git' and command 'git'
root@docker-desktop:/Users/lwaldron/dockerhome/git/hub.bulker.io# echo ${HOME}
/root
root@docker-desktop:/Users/lwaldron/dockerhome/git/hub.bulker.io# ls ${HOME}
root@docker-desktop:/Users/lwaldron/dockerhome/git/hub.bulker.io# 

[lwaldron]

With the shortened templates, I am root user and /root is empty in other images as well.

[nsheff]

Ok, this is really just a general docker question... have you figured out how to map your local user into any docker container, in general? (I mean, outside a bulker context)

[lwaldron]

No, I was hoping you had :D. But I do know that whatever user you are within Docker, reading and changes to mounted volumes occur as your host user. So an approach could be to:

1. mount --volume $HOME:/root, and
2. for --workdir, use pwd but after substituting $HOME with /root

[nsheff]

Well, as far as I understand from people who use docker with MacOS, it just works by default. So I have no idea why it's not working for you -- maybe something to do with how you set up docker? I don't really see how the approach you mention would solve the issue.

@aakrosh has also used bulker successfully on a mac. Aakrosh how do you make sure you're running as the same user inside the container as you are on the host on a mac? have you run into any of these issues?

[nsheff]

(Also just to be clear -- for me, I have no problem mapping the user using the lines I pointed out above, but I'm on a linux host. I don't use MacOS).

[aakrosh]

I have used bulker on linux successfully, both on my local machine and on servers. I have not tried it on my Mac yet...

On Oct 24, 2019, at 8:31 AM, Nathan Sheffield notifications@github.com wrote:

Well, as far as I understand from people who use docker with MacOS, it just works by default. So I have no idea why it's not working for you -- maybe something to do with how you set up docker? I don't really see how the approach you mention would solve the issue.

@aakrosh https://github.com/aakrosh has also used bulker successfully on a mac. Aakrosh how do you make sure you're running as the same user inside the container as you are on the host on a mac? have you run into any of these issues?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/databio/bulker/issues/30?email_source=notifications&email_token=ACDG7B6KPVGWLTZJU26IHU3QQGITZA5CNFSM4JDFRRCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECE3KKA#issuecomment-545895720, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDG7B2F2V6WIQ2UJHJZQZ3QQGITZANCNFSM4JDFRRCA.

[lwaldron]

Nathan, would it be helpful for me to provide you with ssh access to one of my Macs? I can send you instructions offline.

[nsheff]

Sure, I can poke around a bit and see. it would help me understand the problem, anyway. Can't guarantee I can solve it :).

[nsheff]

I eventually solved this on the mac by creating a custom passwd file that just grabbed the one local user, and then mounting that, instead of the global /etc/passwd, which on macs for some reason doesn't contain the active user.

so, instead of: --volume="/etc/passwd:/etc/passwd:ro" \, I have something like: --volume="/template/mac_passwd:/etc/passwd:ro" \

which file looks something like:

USERNAME:x:1000:1000:ns,,,:/home/USERNAME:/bin/bash

USERNAME:x:USERID:GROUPID:ns,,,:/home/USERNAME:/bin/bash

I got this to work in one case, but haven't made it systematic. bulker could detect it's running on a mac and create this user file so the containers can run more smoothly (with user mapped)

[lwaldron]

Just a note that I still see this on 0.5.0-dev; here is a minimal reproducible example on OSX:

Levis-MBP:~ lwaldron$ bulker activate demo
Bulker config: /Users/lwaldron/bulker_config.yaml
Activating bulker crate: demo
bulker/demo| ~$ _cowsay
Starting interactive docker shell for image 'nsheff/cowsay' and command 'cowsay'
I have no name!@docker-desktop:/Users/lwaldron$ whoami
whoami: cannot find name for user ID 503
I have no name!@docker-desktop:/Users/lwaldron$ echo ${HOME}
/
I have no name!@docker-desktop:/Users/lwaldron$ 

[nsheff]

Right -- we never solved this systematically... we had only solved it for your particular setup...

I wonder if bulker could detect that it's running on a mac and automatically do this...

[lwaldron]

How about a little Mac-only script that updates the templates? Even if it has to be run manually, it wouldn’t be a big deal. I just can’t remember now the changes that have to be made. --

Levi Waldron

Associate Professor

Department of Epidemiology and Biostatistics

CUNY Graduate School of Public Health and Health Policy

Institute for Implementation Science in Population Health

55 W 125th St, New York NY 10035

https://waldronlab.io

[nsheff]

Try this:

#!/bin/sh

BULKERDIR=`dirname $BULKERCFG`
cd $BULKERDIR/templates
echo "USERNAME:x:$(id -u):$(id -g):ns,,,:/home/$USER:/bin/bash" > mac_passwd
sed "s|/etc/passwd|$BULKERDIR/templates/mac_passwd|" docker_executable.jinja2 > docker_executable_mac.jinja2
sed -i "s|docker_executable.jinja2|docker_executable_mac.jinja2|" $BULKERCFG

[lwaldron]

I had to provide a backup extension to sed -i in the last line to avoid a syntax error, as follows:

#!/bin/sh
BULKERDIR=`dirname $BULKERCFG`
cd $BULKERDIR/templates
echo "USERNAME:x:$(id -u):$(id -g):ns,,,:/home/$USER:/bin/bash" > mac_passwd
sed "s|/etc/passwd|$BULKERDIR/templates/mac_passwd|" docker_executable.jinja2 > docker_executable_mac.jinja2
sed -i .bak "s|docker_executable.jinja2|docker_executable_mac.jinja2|" $BULKERCFG

I still have the same behavior though:

waldronlab/levi| ~$ bulker load demo
Bulker config: /Users/lwaldron/bulker_config.yaml
That manifest has already been loaded. Overwrite? [y/N] y
Removing all executables in: /Users/lwaldron/bulker_crates/bulker/demo/default
Loading manifest: 'bulker/demo:default'. Activate with 'bulker activate bulker/demo:default'.
Commands available: cowsay, fortune
waldronlab/levi| ~$ bulker activate demo
Bulker config: /Users/lwaldron/bulker_config.yaml
Activating bulker crate: demo
bulker/demo| ~$ cat `which cowsay`
#!/bin/sh

docker run --rm --init \
  -i \
  --user=$(id -u):$(id -g) \
  --network="host" \
  --env "DISPLAY" \
  --volume "$HOME:$HOME" \
  --volume="/etc/group:/etc/group:ro" \
  --volume="/Users/lwaldron/templates/mac_passwd:/etc/passwd:ro" \
  --volume="/etc/shadow:/etc/shadow:ro"  \
  --volume="/etc/sudoers.d:/etc/sudoers.d:ro" \
  --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
  --workdir="`pwd`" \
  nsheff/cowsay cowsay "$@"
bulker/demo| ~$ _cowsay
Starting interactive docker shell for image 'nsheff/cowsay' and command 'cowsay'
I have no name!@docker-desktop:/Users/lwaldron$ whoami
whoami: cannot find name for user ID 503
I have no name!@docker-desktop:/Users/lwaldron$ 

[lwaldron]

I realized that the above script of course modifies the executable template but not the shell template. I had to make some changes to the mac_passwd file, but now the following works for executables and shells! Just remind users to do bulker load after making any changes to the template.

#!/bin/sh                                                                                                                    

BULKERDIR=`dirname $BULKERCFG`
cd $BULKERDIR/templates
echo "$USER:x:$(id -u):$(id -g):ns,,,:$HOME:/bin/bash" > mac_passwd
sed "s|/etc/passwd|$BULKERDIR/templates/mac_passwd|" docker_executable.jinja2 > docker_executable_mac.jinja2
sed -i .bak "s|docker_executable.jinja2|docker_executable_mac.jinja2|" $BULKERCFG
sed "s|/etc/passwd|$BULKERDIR/templates/mac_passwd|" docker_shell.jinja2 > docker_shell_mac.jinja2
sed -i .bak "s|docker_shell.jinja2|docker_shell_mac.jinja2|" $BULKERCFG

For example:

Levis-MBP:~ lwaldron$ bulker load demo
Bulker config: /Users/lwaldron/bulker_config.yaml
That manifest has already been loaded. Overwrite? [y/N] y
Removing all executables in: /Users/lwaldron/bulker_crates/bulker/demo/default
Loading manifest: 'bulker/demo:default'. Activate with 'bulker activate bulker/demo:default'.
Commands available: cowsay, fortune
Levis-MBP:~ lwaldron$ bulker activate demo
Bulker config: /Users/lwaldron/bulker_config.yaml
Activating bulker crate: demo
bulker/demo| ~$ _cowsay
Starting interactive docker shell for image 'nsheff/cowsay' and command 'cowsay'
lwaldron@docker-desktop:~$ whoami
lwaldron
lwaldron@docker-desktop:~$ echo $HOME
/Users/lwaldron

[lwaldron]

Just one more note, I think all the user/group/password mapping is actually unnecessary on OSX; note that even inside the container I still have the OSX home directory. So it would probably be a more parsimonious solution just to remove unnecessary lines from the template, but I tended to get syntax errors when I just started removing or commenting out lines so I gave up since this works.

[nsheff]

but now the following works for executables and shells!

Yep that makes sense, nice!

all the user/group/password mapping is actually unnecessary on OSX

what do you mean, which parts are unnecessary? do you mean parts in this adapter script, or parts in the original templates?

[lwaldron]

all the user/group/password mapping is actually unnecessary on OSX

what do you mean, which parts are unnecessary? do you mean parts in this adapter script, or parts in the original templates?

I would think that all of the following are unnecessary?

  --user=$(id -u):$(id -g) \{% endif %}{% if not pkg.no_network %}
  --volume="/etc/group:/etc/group:ro" \
  --volume="/Users/lwaldron/templates/mac_passwd:/etc/passwd:ro" \
  --volume="/etc/shadow:/etc/shadow:ro"  \
  --volume="/etc/sudoers.d:/etc/sudoers.d:ro" \

[nsheff]

If you just eliminate all that, does it work correctly? In that case we didn't need to do any of the mapping stuff at all... ?

[lwaldron]

Actually the only ones I could get rid of from demo _cowsay and still have the same username and home directory were:

--volume="/etc/group:/etc/group:ro"
--volume="/etc/shadow:/etc/shadow:ro"
--volume="/etc/sudoers.d:/etc/sudoers.d:ro"

It looks like my long list of com.apple.* groups are lost once I'm in a docker container with or without the /etc/group mapping, but I don't miss these groups. So I'm not sure whether the above lines are needed or not, but in any case, they don't seem to be hurting anything. And these lines do seem to be necessary to maintaining username and $HOME:

  --user=$(id -u):$(id -g) \
  --volume="/Users/lwaldron/templates/mac_passwd:/etc/passwd:ro" \

[nsheff]

alright sounds great. I've added this script and an explanation in docs.

[Kmrouj]

I'm also a MacOS user, I had exactly the same issue. I'm not able to deploy?install bulker on my machine (MacOS), could you please tell me which script/explanation (your latest comment) you're referring to. Thank you

[nsheff]

script

explanation

[nsheff]

see also: https://github.com/databio/bulker/commit/89ab8a7e9e71d25999e5c3c3aee725ead2bbc2ae#commitcomment-38202075

[Kmrouj]

Thanks for your reply. Unfortunatly it still not working on my mac. Maybe I'm missing something

[Kmrouj]

Here's the return I got after running again with the modifications mentioned in the previous comments. 'Missing crate: No crates exist' I keep having this return no matter what the changes are (I did not understand the meaning of this error)

[lwaldron]

@Kmrouj it looks like you're having problems earlier on - have you managed to run the cowsay boo example? If not, something needs to be fixed / installed before the OSX username fix will help. Also, I think it would help if you ran lines one at a time so we could see the result of each line being run, rather than running several lines at once.

[Kmrouj]

Hi Levi, No, I did not manage to run the 'cowsay boo'. I think there's something I need to fix before! Here's the return I got after running progressively the commands.

[Kmrouj]

Also, After installing bulker and running the 'pip show -f bulker' command (see image) it looks like bulker is installed! maybe the issue here how I can initialize it! (if this logic is correct)

[lwaldron]

So there's something happening at bulker load demo that's out of my league... hopefully @nsheff has some insight there.

[nsheff]

Hi @Kmrouj, it's complaining that your exe_template doesn't exist. you can try confirming that it exists.

if you follow the instructions in the docs directly, the template should be automatically created in the right place. If you're customizing things, then of course you'll just have to make sure that you point to the exe_template you want to use and it exists.

[nsheff]

@Kmrouj I just released a new version of bulker (0.5.0), with many improvements. You might give a try to just re-installing and re-initializing everything and maybe that will work now.

[Kmrouj]

Hi Nathan. That sounds great. Do you have a link? It's published here ! Thank you

[nsheff]

Do you have a link? It's published here !

What do you mean?

[Kmrouj]

I tried to re-install bulker, but I did not manage. I would like to uninstall it and then install it but I was not able using the command --uninstall ? do you know how I can uninstall bulker? thank you

[nsheff]

Unfortunately I don't have capacity to provide guidance at that level -- you'd be far better off consulting Google to figure out the basics like how to uninstall python packages.