Tried to re-provision staging server, remote bower install error

[mcheck]

So this is not likely how I was supposed to do this, and now I need a way out.

I have the staging server on a VPS and was running bower_components 0.21. It is already provisioned and works fine. I upgrade the local copy with bower update and went to 0.24 Wanted to see the varnish cache fix, so I cap staging genesis:provision. I imagine that was not the right thing (or is it?), since it begins to re-provision the packages as deploy user. But it gets to bower, and poops out. Something in the npm package compare? Log follows below (I snipped out a lot of file listings).

So to get staging to use the new genesis/wordpress bower components, do I issue a cap deploy? Even if no local changes? Or is there a cap task to 'bower update' the remote machine? Thanks!

---

$ cap staging genesis:provision
  * 2013-10-24 16:06:49 executing `production'
    triggering start callbacks for `genesis:provision'
  * 2013-10-24 16:06:49 executing `multistage:ensure'
  * 2013-10-24 16:06:49 executing `genesis:provision'
  * executing "mkdir -p /tmp/mysite.com.2013-10-24.160649 /tmp/mysite.com.2013-10-24.160649/bower_components/genesis-wordpress"
    servers: ["staging.mysite.com"]
    [staging.mysite.com] executing command
    command finished in 149ms
    servers: ["staging.mysite.com"]
 ** scp upload ./bin -> /tmp/mysite.com.2013-10-24.160649/bin
    [staging.mysite.com] ./bin/provision
  * scp upload complete
    servers: ["staging.mysite.com"]
 ** scp upload ./provisioning -> /tmp/mysite.com.2013-10-24.160649/provisioning
    [staging.mysite.com] ./provisioning/files/ssh/id_rsa
    [staging.mysite.com] ./provisioning/files/ssh/id_rsa.pub
    [staging.mysite.com] ./provisioning/group_vars/webservers
    [staging.mysite.com] ./provisioning/local
    [staging.mysite.com] ./provisioning/localhost
    [staging.mysite.com] ./provisioning/production
    [staging.mysite.com] ./provisioning/provision.yml
    [staging.mysite.com] ./provisioning/staging
  * scp upload complete
    servers: ["staging.mysite.com"]
 ** scp upload ./bower_components/genesis-wordpress/provisioning -> /tmp/mysite.com.2013-10-24.160649/bower_components/genesis-wordpress/provisioning
    [staging.mysite.com] ./bower_components/genesis-wordpress/provisioning/provision.yml
    [staging.mysite.com] ./bower_components/genesis-wordpress/provisioning/roles/common/tasks/main.yml

    LOTS OF BOWER_COMPONENTS COPYING OVER JUST FINE

    [staging.mysite.com] ./bower_components/genesis-wordpress/provisioning/roles/wordpress/templates/vhosts/003-production
    [staging.mysite.com] ./bower_components/genesis-wordpress/provisioning/roles/wordpress/vars/main.yml
  * scp upload complete
  * executing "sudo -p 'sudo password: ' /tmp/mysite.com.2013-10-24.160649/bin/provision"
    servers: ["staging.mysite.com"]
    [staging.mysite.com] executing command
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] PLAY [webservers] *************************************************************
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] TASK: [Add Node apt-repository] ***********************************************
 ** [out :: staging.mysite.com] ok: [127.0.0.1]
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] TASK: [Update apt repositories] ***********************************************
 ** [out :: staging.mysite.com] ok: [127.0.0.1]
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] TASK: [Install system packages] ***********************************************
 ** [out :: staging.mysite.com] ok: [127.0.0.1] => (item=build-essential,autoconf,re2c,curl,git-core,python-mysqldb,nodejs)
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] TASK: [Install bower] *********************************************************
 ** [out :: staging.mysite.com] failed: [127.0.0.1] => {"changed": true, "cmd": ["npm", "install", "-g", "bower"], "delta": "0:00:07.525819", "end": "2013-10-24 21:07:22.963039", "item": "", "rc": -9, "start": "2013-10-24 21:07:15.437220"}
 ** [out :: staging.mysite.com] stderr: npm http GET https://registry.npmjs.org/bower
 ** [out :: staging.mysite.com] npm http 304 https://registry.npmjs.org/bower
 ** [out :: staging.mysite.com] npm http GET https://registry.npmjs.org/archy/0.0.2
 ** [out :: staging.mysite.com] npm http GET https://registry.npmjs.org/bower-config

    LOTS OF NPM PACKAGES HERE

 ** [out :: staging.mysite.com] npm http 304 https://registry.npmjs.org/mute-stream
 ** [out :: staging.mysite.com] npm http 304 https://registry.npmjs.org/redeyed
 ** [out :: staging.mysite.com] npm http 304 https://registry.npmjs.org/ansicolors
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] FATAL: all hosts have already failed -- aborting
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] PLAY RECAP ********************************************************************
 ** [out :: staging.mysite.com] to retry, use: --limit @/home/deploy/provision.retry
 ** [out :: staging.mysite.com] 
 ** [out :: staging.mysite.com] 127.0.0.1                  : ok=3    changed=0    unreachable=0    failed=1
 ** [out :: staging.mysite.com] 
    command finished in 27121ms

*** Unable to provision as "deploy"!
*** Try running: $ cap staging genesis:provision -S user=... -S password=...

[ericclemmons]

We had a very similar issue with Bower on the Rackspace Chicago data center. Rackspace was fixing it, so we moved servers in the meantime :)

Anyway, so it looks like you couldn't re-provision because of Bower's issue :( So, we can do a couple things.

1) Retry until Bower works again 2) SSH into the machine (ssh staging genesis:ssh), go to the latest deployment (/var/www/mysite.com/master/current) and run "./bin/provision"). You may also want to run "bower update" from within that project to make sure it works.

When you deploy, "bower update" is automatically ran before turning the new server live.

[mcheck]

OK, trying that. Thanks for the lightning fast response!

• Is there a clue in the logs above (or somewhere I can look) as to what bower is doing wrong?
• Since a new provision (I tried that with a vagrant) works, it is the update to new version of bower that has problem?

Will report back.

[ericclemmons]

It's best to try the same commands on the server directly, since that's all capistrano's doing for us.

If they fail on the server, then we're back to testing locally in the project & seeing if there's a version diff between local & prod, or what.

[mcheck]

All releases on the server give the same error when running sudo ./bin/provision at the bower install

Is there a way to use ansible to start over? Basically reverse all the package installs thus far? Or is the best thing to re-serve a new VPS (using digital ocean) and start over?

[mcheck]

Do you want me to look for a version diff between my local and the remote staging? Not sure what I would be looking for...

[ericclemmons]

What does "bower install" do on the server?

[mcheck]

Oddly nothing, reporting that bower is not installed. I reset the production machine after hacking at this for awhile and re-deployed and it works again. So the problem was with bower and upgrading the components from 2.21 to 2.24? Thanks for trying to help, but I'll close this issue since I re-installed - I think it will only again come up in rare circumstances (hopefully!)