search

mitchellh / vagrant-rackspace

Use Vagrant to manage Rackspace Cloud instances.
MIT License
236 stars 155 forks source link

SSH output from shell provisioner hangs #79

Closed yasin-amadmia-mck closed 10 years ago

yasin-amadmia-mck commented 10 years ago

We use shell provisioner script to install puppet and some other packages via Yum. We noticing that vagrant is able to create/boot a cloud server correctly, RSYNC is also fine and it is able to launch the shell provisioner script. However, it appeared initially that the provisioner script is hanging. On further investigations we found that the script is actually completing fine on the cloud server but due to some reason, vagrant output (from the script) seems to be hanging and vagrant not returning correctly (error or success).

Our vagrant file is as below

ENV['VAGRANT_DEFAULT_PROVIDER'] = 'rackspace' Vagrant.configure("2") do |config|

config.vm.box = "dummy" config.vm.hostname = "web" config.ssh.private_key_path = "/root/id_rsa" config.ssh.username = "root" config.ssh.pty = "true"

config.vm.provider "rackspace" do |rs, override| rs.username = "xxx" rs.api_key = "xxx" rs.flavor = /512MB/ rs.image = /CentOS/ rs.rackspace_region = "lon" rs.server_name = "web1.domain.com" rs.public_key_path = "/root/id_rsa.pub" override.ssh.username = "root" end

config.vm.provision "shell", path: "boot.sh", args: "-r webserver -e production"

config.ssh.shell = "bash -l"

config.ssh.keep_alive = true

config.ssh.forward_agent = false config.ssh.forward_x11 = false

config.vagrant.host = :detect

end

boot.sh is simply a shell file with couple of yum commands.

!/bin/bash

set -x

Install PUPPET from puppetlabs.com repo

yum install -y http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm yum install -y puppet yum install -y git ... ...

The output from Vagrant is as below (just the bottom bit where it hangs)

The strange thing is that it won't always hang here. Sometime it will hang at a stage where yum is downloading the packages. And as mentioned above, the script completes successfully on the destination server, it is just that it is not able to display full output and status via vagrant.

I tried setting / unsetting SSH Keep alive within Vagrant file but no avail.

config.ssh.keep_alive = true

krames commented 10 years ago

@geek876 Thanks for reporting this issue.

A couple quick questions:

  1. What version of CentOS are you using? Have you tried using a different version?
  2. What version of vagrant-rackspace are you using? What version of vagrant-rackspace?
yasin-amadmia-mck commented 10 years ago

1.6.2 Centos x64, not tried on another version but could try it.

  1. Using 0.1.7 plugin version
smashwilson commented 10 years ago

@geek876: I've been trying to replicate this, but the only CentOS 6.2 image I see in LON is for first-gen servers, and vagrant-rackspace 0.1.7 can only boot second-gen ones. Are you sure that's the image you're using? If I use your Vagrantfile, I launch a CentOS 6.5 box, and it completes without hanging for me. Maybe give a newer CentOS image a shot.

My best guess at the moment is that puppet is installing a newer sshd rpm, which is killing the ssh connection that Vagrant is using to provision the server. That's just a guess, though. I'll keep looking :mag: :footprints:

yasin-amadmia-mck commented 10 years ago

I tried it with few other images (centos 5.10, Centos 6.5 and Centos 6.5 PVHVM) and all of them exhibit same behavior. Not sure if it is my connection that is dropping off and causing this issue. I am going to try this from a different connection and see if it improves.

yasin-amadmia-mck commented 10 years ago

One interesting observation. When I do vagrant up --no-provision and then after the VM is up and running, If i do vagrant provision, it never hangs. It is only when I do provisioning and booting the VM at the same time, it hangs.

smashwilson commented 10 years ago

This looks related: mitchellh/vagrant#603. If I'm reading it correctly, there's a chance that the root cause is a Net::SSH bug! I also find it interesting that the original poster there was also installing Puppet on a CentOS image.

Did you install Vagrant with one of the installers? Is there a chance that there's an older, gem-installed version of Vagrant hanging around, using an older Ruby? Try running which vagrant and vagrant --version to check your Vagrant install.

smashwilson commented 10 years ago

It would also be helpful if you could post the full output of running Vagrant with debugging mode enabled to a gist.

smashwilson commented 10 years ago

I'm going to close this now; I suspect it's an upstream problem, and I can't seem to reproduce it myself to be able to track things down. Please re-open if you run into this again and have that debugging output for us!

tacoben commented 9 years ago

I was getting a similar problem with (insert any) provisioners when i was installing an app that blew up on installation. My specific case was postgres.

I resolved the issue by putting MUCH more memory on the base instance.

maxlinc commented 9 years ago

@tacoben that's useful info. Can you share a Vagrantfile that reproduces the problem, and capture vagrant provision --debug to see if it shows what vagrant is doing when SSH hangs?