test-kitchen / kitchen-docker

A Test Kitchen Driver for Docker
Apache License 2.0
462 stars 232 forks source link

Can't run kitchen-docker with Docker for Mac #207

Open chilicheech opened 8 years ago

chilicheech commented 8 years ago

I have the new Docker for Mac installed and I can run docker just fine. When I run kitchen docker though I get the following error:

)-> kitchen converge --log-level=debug
-----> Starting Kitchen (v1.7.3)
D      [kitchen::driver::docker command] BEGIN (sudo -E docker >> /dev/null 2>&1)
D      [kitchen::driver::docker command] END (0m0.01s)
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::UserError
>>>>>> Message: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

D      ------Exception-------
D      Class: Kitchen::UserError
D      Message: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/
D      ---Nested Exception---
D      Class: Kitchen::ShellOut::ShellCommandFailed
D      Message: Expected process to exit with [0], but received '1'
---- Begin output of sudo -E docker >> /dev/null 2>&1 ----
STDOUT:
STDERR:
---- End output of sudo -E docker >> /dev/null 2>&1 ----
Ran sudo -E docker >> /dev/null 2>&1 returned 1
D      ------Backtrace-------
D      /Users/thiagoo/.chefdk/gem/ruby/2.1.0/gems/kitchen-docker-2.4.0/lib/kitchen/driver/docker.rb:78:in `rescue in verify_dependencies'
D      /Users/thiagoo/.chefdk/gem/ruby/2.1.0/gems/kitchen-docker-2.4.0/lib/kitchen/driver/docker.rb:76:in `verify_dependencies'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/driver.rb:45:in `for_plugin'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:230:in `new_driver'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:243:in `new_instance'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:139:in `block in build_instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:138:in `map'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:138:in `with_index'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:138:in `build_instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/config.rb:114:in `instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command.rb:115:in `filtered_instances'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command.rb:145:in `parse_subcommand'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command/action.rb:38:in `block in call'
D      /opt/chefdk/embedded/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/command/action.rb:37:in `call'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/cli.rb:56:in `perform'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/cli.rb:180:in `block (2 levels) in <class:CLI>'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/cli.rb:321:in `invoke_task'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/bin/kitchen:13:in `block in <top (required)>'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/lib/kitchen/errors.rb:154:in `with_friendly_errors'
D      /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.3/bin/kitchen:13:in `<top (required)>'
D      /opt/chefdk/bin/kitchen:17:in `load'
D      /opt/chefdk/bin/kitchen:17:in `<main>'
D      ----------------------

)-> sudo -E docker >> /dev/null 2>&1

)-> echo $?
0

)->
peterabbott commented 8 years ago

You have to make sure that DOCKER_HOST is not set to the tcp address you'd use with Docker Machine or use the socket attribute to set to unix:///var/run/docker.sock

There is an existing issue that captured all the things around getting this driver to work with the native client: https://github.com/portertech/kitchen-docker/issues/201

robertlabrie commented 8 years ago

Just run eval (docker-machine env default) on the shell. This is an issue with the Mac, not the driver. If you can't docker ps then the driver doesn't stand a chance

chilicheech commented 8 years ago

i don't use docker-machine, though.. i'm using the native docker for mac. no docker environment variables need to be set for it.

robertlabrie commented 8 years ago

Sorry, I misunderstood. Good luck!

chilicheech commented 8 years ago

I tried the use_sudo: false workaround from #201 and it works. 😄 .. But it's just a workaround. It should work whether or not I use sudo.

ozbillwang commented 8 years ago

So the work around steps are:

    pinata set network nat external-bind=true
    driver:
      name: docker
      use_sudo: false
erikogan commented 8 years ago

I added a PR (#210) to add an environment variable to override the default value before I noticed this issue.

I don’t want to edit (all of) my .kitchen.yml file(s), because this will break kitchen-docker for others on my team for whom the default value is correct.

I had been editing the files I needed to work on, but it is cumbersome enough to do, and I’m worried I’ll accidentally check it in.

jlruizmlg commented 8 years ago

You can try the docker version 1.12 beta20 to me works

davidcpell commented 7 years ago

Ran into this issue today:

OSX 10.11.6 Docker version 1.12.1, build 6f9534c kitchen-docker 2.6.0

Had to add use_sudo: false to .kitchen.yml to get it to work.

erikogan commented 7 years ago

@davidcpell You can also add use_sudo: false to your ~/.kitchen/config.yml in the driver section.

This solution was mentioned on the PR referenced above, but never brought back here.

davidcpell commented 7 years ago

Thanks @erikogan! Came back here today because I ran into the issue again and forgot how I had gotten around it. That will help a bunch.

walterdolce commented 7 years ago

I'm having the very same issue on one of the machines I work from. The user from which I run bundle exec kitchen list (which fails) triggers sudo -E docker >> /dev/null 2>&1 under the hood (as seen from the debug output).

If I run the same command from the CLI I am prompted to enter the password. This is what I suspect it's causing the exception to be raised. Running the same command without sudo works as expected.

walterdolce commented 7 years ago

FYI (Please note I purposefully removed >> {NIL|/dev/null})

$ bundle exec kitchen list -l debug
D      [local command] BEGIN (sudo -E docker 2>&1)
       sudo: no tty present and no askpass program specified
D      [local command] END (0m0.08s)
...
docwhat commented 7 years ago

See @erikogan's solution below... You have to tell kitchen not to use sudo.

peterabbott commented 7 years ago

The main issue is probably the check for remote_socket forcing sudo, https://github.com/test-kitchen/kitchen-docker/blob/master/lib/kitchen/driver/docker.rb#L60

When using Docker for Mac (or Windows) the DOCKER_HOST environment variable no longer needs to be set. So when the driver starts it sees that it is running locally and assumes that it needs to run sudo. This behaviours could perhaps be reviewed or the check made a bit smarter???

coderanger commented 7 years ago

Yeah, the right long-term fix is probably to use docker version or docker info to probe things a bit more.

joegoggins commented 7 years ago

Is there anything I could do to help address this issue?

Or perhaps there is a different recommendation for using Test Kitchen with Docker on a Mac?

I'm blocked right now and I'm not sure how to proceed.

If I use this use_sudo: false work-around, I can make the Class: Kitchen::UserError Message: You must first install the Docker CLI tool... error go away, however, instead I get a failure like this on kitchen converge:

            Installing chef 12.14.89
                 installing with dpkg...
                 Selecting previously unselected package chef.
          (Reading database ... 16043 files and directories currently installed.)
                 Preparing to unpack .../chef_12.14.89-1_amd64.deb ...
                 Unpacking chef (12.14.89-1) ...
                 Setting up chef (12.14.89-1) ...
                 Thank you for installing Chef!
                 mkdir: cannot create directory ‘/tmp/kitchen’: No space left on device

which makes sense. When you run kitchen login default as the kitchen user (not root) and run mkdir /tmp/kitchen you get this same error, it's because the provisioner now no longer has privileges to do this.

How do I give the provisioner the permissions it needs when use_sudo: false?

My .kitchen.yml

---
driver:
  name: docker
  use_sudo: false
  privileged: true
transport:
  max_ssh_sessions: 6
provisioner:
  name: chef_zero
  require_chef_omnibus: 12.21.4
  log_level: info
  data_bags_path: test/data_bags
  encrypted_data_bag_secret_key_path: test/data_bags/encrypted_data_bag_secret
verifier:
  name: inspec
  format: documentation
platforms:
  # We run this special image so we can keep upstart running
  # inside the container because
  # - runit services inside the container require it
  # - Gets around SSH connect issue: https://github.com/test-kitchen/kitchen-docker/issues/254
  # - runit cookbook recommends it: https://github.com/chef-cookbooks/runit#development
  - name: ubuntu-14.04
    driver_config:
      image: ubuntu-upstart:14.04
      platform: ubuntu
      disable_upstart: false
      run_command: /sbin/init

suites:
  - name: default
    named_run_list: default
    verifier:
      inspec_tests:
        - path: test/smoke/default
        - name: particle-inspec-foo-service
          git: git@github.com:spark/particle-inspec.git
    attributes:
      particle_foo_service:
        my_attribute: "foo"
      tags:
      - my_special_tag
coderanger commented 7 years ago

@joegoggins Please include your full config file. You might have disabled too much sudo.

joegoggins commented 7 years ago

@coderanger I added my failing .kitchen.yml to the above comment. Thanks for the ping asking for this. Let me know if I can provide anything else or what you'd recommend my next step to be!

FilBot3 commented 6 years ago

I also have this issue, but not just for OSX. Even though the default is false for use_sudo, I still had to specify the use_sudo: false option otherwise I got this error:

$ chef exec kitchen converge
-----> Starting Kitchen (v1.17.0)
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::UserError
>>>>>> Message: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
joegoggins commented 6 years ago

On OSX, I was able to get around this error permanently by:

  1. Clicking Docker Icon on the menu bar
  2. Clicking preferences
  3. Clicking "Reset"
  4. Clicking "Remove all data"
  5. Clicking "Reset to factory defaults"

I haven't seen it since.

Not everyone can do this, because it's highly destructive operation, but I suspect most can because they are using Docker purely for dev purposes on their workstation.

Dispader commented 6 years ago

Thanks, @joegoggins — that worked for me, here.

JoshuaCWebDeveloper commented 6 years ago

I faced the same error on Windows 10 and came across this issue just by Googling the underlying Ruby exception. Even though my error is of a different nature than what is described here, I wanted to chime in and add that the recommended solution of specifying use_sudo: false worked for me as well to get past this error.

In my case, the error was because I am running kitchen under MINGW64 on Git Bash for Windows, and there is no sudo command on this shell, so running sudo … anything results in an error.

detj commented 5 years ago

Looks like this is still an issue. Couldn't get this to work for the life of me. Running docker v18.06.1-ce on macOS 10.13.6.

.kitchen.yml looks like this

---
driver:
  name: docker
  use_sudo: false
  privileged: true

I tried @joegoggins's solution of restoring Docker to its factory settings, removing all data, but no luck. Tried all combinations of use_sudo and privileged and other settings too. Getting this error still

>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 2 actions failed.
>>>>>>     Failed to complete #create action: [Expected process to exit with [0], but received '125'
---- Begin output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultubuntu1604-detj-pinwheellocal-da42cs3y --privileged fa5c9ca31d03 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
STDOUT:
STDERR: docker: you are not authorized to perform this operation: server returned 401.
See 'docker run --help'.
---- End output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultubuntu1604-detj-pinwheellocal-da42cs3y --privileged fa5c9ca31d03 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
Ran docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultubuntu1604-detj-pinwheellocal-da42cs3y --privileged fa5c9ca31d03 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid returned 125] on default-ubuntu-1604
>>>>>>     Failed to complete #create action: [Expected process to exit with [0], but received '125'
---- Begin output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultcentos7-detj-pinwheellocal-b07r76rl --privileged aee8aabc8079 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
STDOUT:
STDERR: docker: you are not authorized to perform this operation: server returned 401.
See 'docker run --help'.
---- End output of docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultcentos7-detj-pinwheellocal-b07r76rl --privileged aee8aabc8079 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid ----
Ran docker -H unix:///var/run/docker.sock run -d -p 22 --name defaultcentos7-detj-pinwheellocal-b07r76rl --privileged aee8aabc8079 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid returned 125] on default-centos-7
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Running kitchen diagnose --all does pass though, which is all the more confusing.

Is there any progress with this issue ?

martinm82 commented 5 years ago

@detj how does your docker run command looks like? In my case I got it working with Docker 18.6.1 (on macOS 10.13.6) with the following command:

docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):/app -w /app salt-testing sh

and the use_sudo: false setting in kitchen.yml.

In case your image does not run as root then you have to pass --group-add root in order to get read permissions on /var/run/docker.sock within the container, otherwise you will get Permission denied errors.

darkn3rd commented 5 years ago

I am able to use kitchen on Mac OS X with either Docker-For-Mac (hyperkit) or Docker-Machine (virtualbox).

I have noticed that sometime, the ssh key is set with bad permissions, which I have to run chmod to reset them so that kitchen login works.

luisra51 commented 11 months ago

This is working for me sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock