docker-machine create is hanging with driver virtualbox : SSH cmd err, output: exit status 255:

[opskumu]

# docker -v && docker-machine -v && docker-compose -v
Docker version 1.6.2, build ba1f6c3/1.6.2
docker-machine version 0.3.1 (40ee236)
docker-compose version: 1.3.3
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
# rpm -qa | grep virtual -i
VirtualBox-4.3-4.3.30_101610_el7-1.x86_64

docker-machine create is hanging，the debug logs：

Getting to WaitForSSH function...
Testing TCP connection to: localhost:45105
Using SSH client type: external
About to run SSH command:
exit 0
&{/usr/bin/ssh [/usr/bin/ssh -o PasswordAuthentication=no -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i /root/.docker/machine/machines/dev2/id_rsa -p 45105 docker@localhost exit 0] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}
SSH cmd err, output: exit status 255: 
Error getting ssh command 'exit 0' : exit status 255
Getting to WaitForSSH function...
Testing TCP connection to: localhost:45105
Using SSH client type: external
About to run SSH command:
exit 0
&{/usr/bin/ssh [/usr/bin/ssh -o PasswordAuthentication=no -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i /root/.docker/machine/machines/dev2/id_rsa -p 45105 docker@localhost exit 0] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>}
SSH cmd err, output: exit status 255: 
Error getting ssh command 'exit 0' : exit status 255
... ...

[chrisbenson]

@nathanleclaire Sorry for the delay getting back to you. I temporarily gave up and didn't check back quickly. I'll be more vigilant waiting for your replies going forward.

It's not the ~/.ssh/config, because I actually deleted mine to just make sure it wasn't that. It had no effect.

After I run docker-machine --debug create -d virtualbox default, which generates the afformentioned error over and over again, and then I Ctrl + C out of it.

Then I enter docker-machine ssh default -vvv, and get this:

OpenSSH_7.2p2, OpenSSL 1.0.2g 1 Mar 2016 debug1: Reading configuration data /usr/local/etc/ssh/ssh_config debug2: resolving "127.0.0.1" port 22 debug2: ssh_connect_direct: needpriv 0 debug1: Connecting to 127.0.0.1 [127.0.0.1] port 22. debug2: fd 3 setting O_NONBLOCK debug1: connect to address 127.0.0.1 port 22: Connection refused debug1: Trying again... debug1: Connecting to 127.0.0.1 [127.0.0.1] port 22. debug2: fd 3 setting O_NONBLOCK debug1: connect to address 127.0.0.1 port 22: Connection refused debug1: Trying again... debug1: Connecting to 127.0.0.1 [127.0.0.1] port 22. debug2: fd 3 setting O_NONBLOCK debug1: connect to address 127.0.0.1 port 22: Connection refused ssh: connect to host 127.0.0.1 port 22: Connection refused exit status 255

[nathanleclaire]

@chrisbenson Just to make sure, what's the contents of /usr/local/etc/ssh/ssh_config?

[chrisbenson]

@nathanleclaire Here's the result of 'cat /usr/local/etc/ssh/ssh_config':

# This is the ssh client system-wide configuration file.  See
# ssh_config(5) for more information.  This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.

# Configuration data is parsed as follows:
#  1. command line options
#  2. user-specific file
#  3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.

# Site-wide defaults for some commonly used options.  For a comprehensive
# list of available options, their meanings and defaults, please see the
# ssh_config(5) man page.

# Host *
#   ForwardAgent no
#   ForwardX11 no
#   RhostsRSAAuthentication no
#   RSAAuthentication yes
#   PasswordAuthentication yes
#   HostbasedAuthentication no
#   GSSAPIAuthentication no
#   GSSAPIDelegateCredentials no
#   BatchMode no
#   CheckHostIP yes
#   AddressFamily any
#   ConnectTimeout 0
#   StrictHostKeyChecking ask
#   IdentityFile ~/.ssh/identity
#   IdentityFile ~/.ssh/id_rsa
#   IdentityFile ~/.ssh/id_dsa
#   Port 22
#   Protocol 2,1
#   Cipher 3des
#   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc
#   MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160
#   EscapeChar ~
#   Tunnel no
#   TunnelDevice any:any
#   PermitLocalCommand no
#   VisualHostKey no
#   ProxyCommand ssh -q -W %h:%p gateway.example.com
#   RekeyLimit 1G 1h

[nathanleclaire]

Hm, OK, so nothing's set. Can you try the 0.7.0 release to see if you fare any better (it's brand new, so you'll have to download the binaries as Toolbox hasn't been baked yet)

[chrisbenson]

@nathanleclaire I tried 0.7.0, but it made no difference. Getting exactly the same error. Any other ideas? Thanks!

[nathanleclaire]

@chrisbenson Does --native-ssh work for you, or does it exhibit the same issue?

[chrisbenson]

@nathanleclaire It exhibits the same issue, Nathan.

[nathanleclaire]

@chrisbenson You have any kind of firewall / VPN / proxy on?

[chrisbenson]

@nathanleclaire No sir, not a thing. I'm working on a modern, completely updated MacBook Pro running El Capitan. I keep everything (Docker, Docker Machine, OpenSSL, OpenSSH) updated with Homebrew, except I installed VirtualBox using their download binary. There is nothing remarkable about my setup, which is why I'm stumped.

[nathanleclaire]

@chrisbenson Couple things to please try:

• If you use the provided Machine binary from the releases page, does it make any difference? There is a very small chance there could potentially be a cgo vs. native Go issue due to use of Homebrew.
• If you look at the exposed VirtualBox port in the GUI (forwarded port 22 in Network Settings for the VM), can you SSH into it manually? (e.g. ssh docker@127.0.0.1 -p 34567 -- password is tcuser)
  • If not, does stuff like ping 127.0.0.1 work?
• Is your Mac system firewall enabled? (in Settings IIRC)
• Please take a look at the VirtualBox logs (nested inside of ~/.docker/machine/machines/default/default) to see if there is anything interesting, e.g. errors.
• 127.0.0.1 is your default loopback, right? what does your /etc/hosts file look like?

[komocode]

--native-ssh works for me, but this problem occurs everytime I start a new terminal session in OS X. I've had nothing but problems with docker-machine in production, I hope the upcoming Docker for Mac solves these issues.

[ccamp46]

This is happening on windows 7, latest docker toolbox, when i put the pc to sleep at night. When I wake the PC all docker-machine commands time out, but i can docker-machine ssh $node just fine. Inside the nodes, docker is running fine and containers i launched the previous day are still running.

$ docker -v && docker-machine -v Docker version 1.12.0, build 8eab29e docker-machine.exe version 0.8.0, build b85aac1

[angotv]

$ docker-machine env

$ docker-machine regenerate-certs [machine name]

$ docker-machine ls ok!

[CollinChaffin]

Wow three YEARS later on WIN7x64 here and this is STILL HAPPENING!!! Isn't it time to fix it or just call the product broken when it's IMPOSSIBLE to issue a simple 1st default create command due to this?

(default) DBG | About to run SSH command:
(default) DBG | exit 0
(default) DBG | SSH cmd err, output: exit status 255:
(default) DBG | Error getting ssh command 'exit 0' : ssh command error:
(default) DBG | command : exit 0
(default) DBG | err     : exit status 255

[tenorok]

Unfortunately I have the same issue. It seems I read all links in google and nothing can fix the problem. Please help!

Is it correct to create machines inside VPS like DigitalOcean droplet? I have the problem on the such server:

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:    18.04
Codename:   bionic

# docker -v
Docker version 18.06.1-ce, build e68fc7a

# docker-machine -v
docker-machine version 0.15.0, build b48dc28d

# VBoxManage --version
5.2.10_Ubuntur121806

# docker-machine --debug create --driver virtualbox mymachine

...

(mymachine) Waiting for an IP...
(mymachine) DBG | STDOUT:
(mymachine) DBG | {
(mymachine) DBG | Waiting for VM "mymachine" to power on...
(mymachine) DBG | VM "mymachine" has been successfully started.
(mymachine) DBG | }
(mymachine) DBG | STDERR:
(mymachine) DBG | {
(mymachine) DBG | }
(mymachine) DBG | Checking vm logs: /root/.docker/machine/machines/mymachine/mymachine/Logs/VBox.log
(mymachine) DBG | Getting to WaitForSSH function...
(mymachine) DBG | Using SSH client type: external
(mymachine) DBG | Using SSH private key: /root/.docker/machine/machines/mymachine/id_rsa (-rw-------)
(mymachine) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@127.0.0.1 -o IdentitiesOnly=yes -i /root/.docker/machine/machines/mymachine/id_rsa -p 44643] /usr/bin/ssh <nil>}
(mymachine) DBG | About to run SSH command:
(mymachine) DBG | exit 0

# And then many times:

(mymachine) DBG | SSH cmd err, output: exit status 255:
(mymachine) DBG | Error getting ssh command 'exit 0' : ssh command error:
(mymachine) DBG | command : exit 0
(mymachine) DBG | err     : exit status 255
(mymachine) DBG | output  :
(mymachine) DBG | Getting to WaitForSSH function...
(mymachine) DBG | Using SSH client type: external
(mymachine) DBG | Using SSH private key: /root/.docker/machine/machines/mymachine/id_rsa (-rw-------)
(mymachine) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@127.0.0.1 -o IdentitiesOnly=yes -i /root/.docker/machine/machines/mymachine/id_rsa -p 44643] /usr/bin/ssh <nil>}
(mymachine) DBG | About to run SSH command:
(mymachine) DBG | exit 0

I tried create ssh config:

# cat ~/.ssh/config
Host *
  TCPKeepAlive yes
  ServerAliveInterval 10
  ServerAliveCountMax 10
  ForwardAgent yes
  ControlMaster no

Using --native-ssh no solve issue.

[nathanleclaire]

@tenorok You can't nest virtualization in most cloud providers which is what attempting to create a VirtualBox machine on DO would be doing.

[beatwade]

I've tried the tool-box and meet the same issues when I used cmd and native ssh (windows oponssl) and resolved this issues by tring ssh from git /usr/bin/ssh.

[mirr254]

Getting this issue in 2020.

&{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@159.203.168.200 -p 22] /usr/local/bin/ssh <nil>}
About to run SSH command:
exit 0
SSH cmd err, output: exit status 255: 
Error getting ssh command 'exit 0' : ssh command error:
command : exit 0
err     : exit status 255
output  : 
Getting to WaitForSSH function...
(dvc1) Calling .GetSSHHostname
(dvc1) Calling .GetSSHPort
(dvc1) Calling .GetSSHKeyPath
(dvc1) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@159.203.168.200 -p 22] /usr/local/bin/ssh <nil>}
About to run SSH command:
exit 0

➜  dogvscat git:(master) ✗ docker-machine -v
docker-machine version 0.16.2, build bd45ab1
➜  dogvscat git:(master) ✗ docker version
Client: Docker Engine - Community
 Version:           19.03.8