machine-drivers / docker-machine-driver-xhyve

docker-machine/minikube/minishift driver plugin for xhyve/hyperkit (native macOS hypervisor.framework)
https://godoc.org/github.com/machine-drivers/docker-machine-driver-xhyve
BSD 3-Clause "New" or "Revised" License
889 stars 73 forks source link

VM won't start without a TTY #141

Open TomFrost opened 7 years ago

TomFrost commented 7 years ago

Running macOS Sierra. Installed docker-machine (0.8.1) and docker-machine-driver-xhyve (0.2.3) from brew. Chown'ed to root and set the setuid bit.

docker-machine start default in a terminal works great. The machine comes up wonderfully, volumes mount via NFS.

However, starting the machine through a LaunchAgent fails. The only difference visible through ps ax is that launchctl obviously doesn't map a TTY to the process. Here's my com.docker.machine.default.plist for reproduction:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>EnvironmentVariables</key>
    <dict>
      <key>PATH</key>
      <string>/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
    </dict>
    <key>Label</key>
    <string>com.docker.machine.default</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/local/bin/docker-machine</string>
      <string>start</string>
      <string>default</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/tmp/dm_err.log</string>
    <key>StandardOutPath</key>
    <string>/tmp/dm_out.log</string>
  </dict>
</plist>

Save that to ~/Library/LaunchAgents and load with launchctl load ~/Library/LaunchAgents/com.docker.machine.default.plist. Verify that xhyve is running with the correct parameters with ps ax | grep xhyve. The following gets dumped to /tmp/dm_out.log:

Starting "default"...
(devbox) Waiting for VM to come online...
(devbox) Waiting on a pseudo-terminal to be ready... done
(devbox) Hook up your terminal emulator to /dev/ttys002 in order to connect to your VM

Execution hangs at that point. /tmp/dm_err.log has no data.

That plist successfully launches the VM on other drivers, but I'm guessing this one requires an active TTY for some reason. Any ideas?

TomFrost commented 7 years ago

Following up for anyone having the same issue; to allocate a pseudo-tty to this process, I installed tmux and used the following plist file. This works a treat (logging is removed in this one, add it back if you like):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>EnvironmentVariables</key>
    <dict>
      <key>PATH</key>
      <string>/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
    </dict>
    <key>Label</key>
    <string>com.docker.machine.default</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/local/bin/tmux</string>
      <string>-c</string>
      <string>/usr/local/bin/docker-machine start default; exit</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
  </dict>
</plist>

It would still be ideal to be able to launch xhyve without a tty though.

im-kulikov commented 7 years ago

@TomFrost thats not working for me..

(dev) DBG | Getting to WaitForSSH function...
(dev) DBG | Using SSH client type: external
(dev) DBG | {[-o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker@192.168.64.14 -o IdentitiesOnly=yes -i /Users/<username>/.docker/machine/machines/dev/id_rsa -p 22] /usr/bin/ssh}
(dev) DBG | About to run SSH command:
(dev) DBG | exit 0
(dev) DBG | SSH cmd err, output: exit status 255:
(dev) DBG | Error getting ssh command 'exit 0' : Something went wrong running an SSH command!
(dev) DBG | command : exit 0
(dev) DBG | err     : exit status 255
(dev) DBG | output  :
(dev) DBG |
(dev) DBG | Getting to WaitForSSH function...
(dev) DBG | Using SSH client type: external
(dev) DBG | {[-o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker@192.168.64.14 -o IdentitiesOnly=yes -i /Users/<username>/.docker/machine/machines/dev/id_rsa -p 22] /usr/bin/ssh}
(dev) DBG | About to run SSH command:
(dev) DBG | exit 0