wuj21 commented 11 years ago

Hi guys,

I am trying to deploy CloudFoundry in vSphere environment, but I was blocked at micro bosh installation. I installed ruby environment following http://docs.cloudfoundry.com/docs/running/bosh/setup/ and downloaded bosh_cli_plugin_micro and downloaded micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz. But micro cannot be installed with "bosh micro deploy ~/stemcells/micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz"

Here is the standard output

ubuntu-bosh:~/deployments$ bosh micro deploy ~/stemcells/micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz Deploying new micro BOSH instance micro01/micro_bosh.yml' tohttps://10.117.38.171:25555' (type 'yes' to continue): yes

Verifying stemcell... File exists and readable OK Using cached manifest... Stemcell properties OK

Stemcell info

Name: micro-bosh-stemcell Version: 1.5.0.pre2

Deploy Micro BOSH Unpacking stemcell | | 0/11 00:00:10 ETA: --:--:--at depth 0 - 20: unable to get local issuer certificate unpacking stemcell (00:00:10)
Uploading stemcell |oo | 1/11 00:00:28 ETA: 00:01:24at depth 0 - 20: unable to get local issuer certificate Uploading stemcell |oo | 1/11 00:01:10 ETA: 00:00:42at depth 0 - 20: unable to get local issuer certificate uploading stemcell (00:01:49)
Creating VM from... |oooo | 2/11 00:03:34 ETA: 00:07:25at depth 0 - 20: unable to get local issuer certificate Creating VM from... |oooo | 2/11 00:03:37 ETA: 00:07:22at depth 0 - 20: unable to get local issuer certificate Creating VM from... |oooo | 2/11 00:03:38 ETA: 00:07:21at depth 0 - 20: unable to get local issuer certificate creating VM from sc-ce581eb9-46d3-4d29-98c1-8c4d39914ab6 (00:01:47)
Waiting for the agent |oooooo | 3/11 00:04:09 ETA: 00:09:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:05:09 ETA: 00:08:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:06:10 ETA: 00:07:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:07:10 ETA: 00:06:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:08:10 ETA: 00:05:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:09:10 ETA: 00:04:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:10:10 ETA: 00:03:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:11:10 ETA: 00:02:43at depth 0 - 20: unable to get local issuer certificate Waiting for the agent |oooooo | 3/11 00:12:10 ETA: 00:01:43at depth 0 - 20: unable to get local issuer certificate

I checked the log files bosh_micro_deploy.log, seems cannot connect to bosh agent. I logged in the VM created by bosh with "vcap/c1oudc0w", and I found the network is not configured correctly. see the attachment, eth0 was not up screen shot 2013-07-04 at 3 53 34 pm

wuj21 commented 11 years ago

Here is the log file of bosh_micro_deploy.log

I, [2013-07-04T02:09:28.185232 #10409] [createstemcell(/tmp/d20130704-10409-13f3d0h/sc-20130704-10409-169dur1/image, )] INFO -- : Generated name: sc-2ca06601-49c0-4948-a3e0-ad41e1209e23 D, [2013-07-04T02:09:28.228964 #10409] [createstemcell(/tmp/d20130704-10409-13f3d0h/sc-20130704-10409-169dur1/image, )] DEBUG -- : Datastores - ephemeral: [], persistent: [], shared: ["boshdatastore"]. I, [2013-07-04T02:09:54.743936 #10485] [0x4c6aff8] INFO -- : No existing deployments found (will save to /home/jackwu/deployments/bosh-deployments.yml) I, [2013-07-04T02:10:03.727515 #10485] [0x4c6aff8] INFO -- : Loading yaml from /tmp/d20130704-10485-zcf6cc/sc-20130704-10485-166q1zm/stemcell.MF I, [2013-07-04T02:10:03.825831 #10485] [createstemcell(/tmp/d20130704-10485-zcf6cc/sc-20130704-10485-166q1zm/image, )] INFO -- : Extracting stemcell to: /tmp/d20130704-10485-zcf6cc/d20130704-10485-14qb1qb I, [2013-07-04T02:10:11.733031 #10485] [createstemcell(/tmp/d20130704-10485-zcf6cc/sc-20130704-10485-166q1zm/image, )] INFO -- : Generated name: sc-4c1b4ce9-8e42-42df-8553-f86d31ed0303 D, [2013-07-04T02:10:11.780346 #10485] [createstemcell(/tmp/d20130704-10485-zcf6cc/sc-20130704-10485-166q1zm/image, )] DEBUG -- : Datastores - ephemeral: [], persistent: [], shared: ["boshdatastore"]. I, [2013-07-04T02:11:16.942744 #10567] [0x43e9000] INFO -- : No existing deployments found (will save to /home/jackwu/deployments/bosh-deployments.yml) I, [2013-07-04T02:11:29.639997 #10567] [0x43e9000] INFO -- : Loading yaml from /tmp/d20130704-10567-luipax/sc-20130704-10567-122rcft/stemcell.MF I, [2013-07-04T02:11:29.684722 #10567] [createstemcell(/tmp/d20130704-10567-luipax/sc-20130704-10567-122rcft/image, )] INFO -- : Extracting stemcell to: /tmp/d20130704-10567-luipax/d20130704-10567-jb4u78 I, [2013-07-04T02:11:36.413007 #10567] [createstemcell(/tmp/d20130704-10567-luipax/sc-20130704-10567-122rcft/image, )] INFO -- : Generated name: sc-c230b78d-8752-4d81-b89b-d461c790c093 D, [2013-07-04T02:11:36.502839 #10567] [createstemcell(/tmp/d20130704-10567-luipax/sc-20130704-10567-122rcft/image, )] DEBUG -- : Datastores - ephemeral: [], persistent: [], shared: ["boshdatastore"]. I, [2013-07-04T02:12:09.460736 #10643] [0x4caf810] INFO -- : No existing deployments found (will save to /home/jackwu/deployments/bosh-deployments.yml) I, [2013-07-04T02:12:27.091501 #10643] [0x4caf810] INFO -- : Loading yaml from /tmp/d20130704-10643-zgespb/sc-20130704-10643-1vp475x/stemcell.MF I, [2013-07-04T02:12:27.192277 #10643] [createstemcell(/tmp/d20130704-10643-zgespb/sc-20130704-10643-1vp475x/image, )] INFO -- : Extracting stemcell to: /tmp/d20130704-10643-zgespb/d20130704-10643-1dsnnu I, [2013-07-04T02:12:39.235446 #10643] [createstemcell(/tmp/d20130704-10643-zgespb/sc-20130704-10643-1vp475x/image, )] INFO -- : Generated name: sc-f567d639-d36a-44ea-8162-8a0aa2664c1d D, [2013-07-04T02:12:39.281164 #10643] [createstemcell(/tmp/d20130704-10643-zgespb/sc-20130704-10643-1vp475x/image, )] DEBUG -- : Datastores - ephemeral: [], persistent: [], shared: ["boshdatastore"]. I, [2013-07-04T02:25:18.841612 #10805] [0x502fff8] INFO -- : No existing deployments found (will save to /home/jackwu/deployments/bosh-deployments.yml) I, [2013-07-04T02:25:32.745767 #10805] [0x502fff8] INFO -- : Loading yaml from /tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/stemcell.MF I, [2013-07-04T02:25:32.844426 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Extracting stemcell to: /tmp/d20130704-10805-xt8ixp/d20130704-10805-16fwde1 I, [2013-07-04T02:25:44.424341 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Generated name: sc-71063669-6aeb-4852-a485-59e91682c16a D, [2013-07-04T02:25:44.470908 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] DEBUG -- : Datastores - ephemeral: [], persistent: [], shared: ["boshdatastore"]. D, [2013-07-04T02:25:44.569859 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] DEBUG -- : cfcluster ephemeral disk bound D, [2013-07-04T02:25:44.569967 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] DEBUG -- : Score: cfcluster: 3108 I, [2013-07-04T02:25:44.570094 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Deploying to: <[Vim.ClusterComputeResource] domain-c1139> / <[Vim.Datastore] datastore-1142> I, [2013-07-04T02:25:44.982111 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Waiting for NFC lease I, [2013-07-04T02:25:48.008524 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Uploading I, [2013-07-04T02:25:48.017230 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Uploading disk to: https://10.117.38.13/nfc/522b20ad-8d65-a43b-0077-c477432f2992/disk-0.vmdk I, [2013-07-04T02:27:13.078386 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Removing NICs I, [2013-07-04T02:27:14.116548 #10805] [createstemcell(/tmp/d20130704-10805-xt8ixp/sc-20130704-10805-z5xx76/image, )] INFO -- : Taking initial snapshot D, [2013-07-04T02:27:17.451387 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] DEBUG -- : Datastores - ephemeral: [], persistent: [], shared: ["boshdatastore"]. D, [2013-07-04T02:27:17.481398 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] DEBUG -- : cfcluster memory bound D, [2013-07-04T02:27:17.481500 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] DEBUG -- : Score: cfcluster: 23 I, [2013-07-04T02:27:17.481651 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Creating vm: vm-5a68f0c7-130d-47d4-a1dc-473713a7ea7b on <[Vim.ClusterComputeResource] domain-c1139> stored in <[Vim.Datastore] datastore-1142> I, [2013-07-04T02:27:17.490792 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Stemcell lives on a different datastore, looking for a local copy of: sc-71063669-6aeb-4852-a485-59e91682c16a. I, [2013-07-04T02:27:17.494230 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Cluster doesn't have stemcell sc-71063669-6aeb-4852-a485-59e91682c16a, replicating I, [2013-07-04T02:27:17.497590 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Replicating sc-71063669-6aeb-4852-a485-59e91682c16a (<[Vim.VirtualMachine] vm-1144>) to sc-71063669-6aeb-4852-a485-59e91682c16a / datastore-1142 I, [2013-07-04T02:28:55.661323 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Replicated sc-71063669-6aeb-4852-a485-59e91682c16a (<[Vim.VirtualMachine] vm-1144>) to sc-71063669-6aeb-4852-a485-59e91682c16a / datastore-1142 (<[Vim.VirtualMachine] vm-1146>) I, [2013-07-04T02:28:55.661462 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Creating initial snapshot for linked clones on <[Vim.VirtualMachine] vm-1146> I, [2013-07-04T02:28:58.696067 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Created initial snapshot for linked clones on <[Vim.VirtualMachine] vm-1146> I, [2013-07-04T02:28:58.696197 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Using stemcell VM: <[Vim.VirtualMachine] vm-1146> I, [2013-07-04T02:28:58.721539 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Cloning vm: <[Vim.VirtualMachine] vm-1146> to vm-5a68f0c7-130d-47d4-a1dc-473713a7ea7b I, [2013-07-04T02:29:13.478713 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Setting VM env: {"vm"=>{"name"=>"vm-5a68f0c7-130d-47d4-a1dc-473713a7ea7b", "id"=>"vm-1148"}, "agent_id"=>"bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d", "networks"=> {"bosh"=> {"cloud_properties"=>{"name"=>"VM Network"}, "netmask"=>"255.255.255.0", "gateway"=>"10.117.38.253", "ip"=>"10.117.38.171", "dns"=>["10.117.0.1", "10.117.0.2"], "type"=>nil, "default"=>["dns", "gateway"], "mac"=>"00:50:56:ae:5f:18"}}, "disks"=>{"system"=>0, "ephemeral"=>1, "persistent"=>{}}, "ntp"=>["10.112.64.1"], "blobstore"=> {"provider"=>"local", "options"=>{"blobstore_path"=>"/var/vcap/micro_bosh/data/cache"}}, "mbus"=>"https://vcap:b00tstrap@0.0.0.0:6868", "env"=>{"bosh"=>{"password"=>nil}}}

I, [2013-07-04T02:29:17.628951 #10805] [create_vm(bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d, ...)] INFO -- : Powering on VM: <[Vim.VirtualMachine] vm-1148> (vm-5a68f0c7-130d-47d4-a1dc-473713a7ea7b) D, [2013-07-04T02:29:20.849545 #10805] [set_vm_metadata(vm-5a68f0c7-130d-47d4-a1dc-473713a7ea7b, ...)] DEBUG -- : Datastores - ephemeral: [], persistent: [], shared: ["boshdatastore"]. D, [2013-07-04T02:29:20.909873 #10805] [0x502fff8] DEBUG -- : Waiting for agent to be ready: nil D, [2013-07-04T02:29:24.908597 #10805] [0x502fff8] DEBUG -- : Waiting for agent to be ready: #<Bosh::Agent::Error: Request details: uri: https://vcap:b00tstrap@10.117.38.171:6868/agent payload: {"method":"ping","arguments":[],"reply_to":"bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d"} user: vcap password: b00tstrap Errno::EHOSTUNREACH: No route to host - connect(2) (https://10.117.38.171:6868)> D, [2013-07-04T02:29:27.908473 #10805] [0x502fff8] DEBUG -- : Waiting for agent to be ready: #<Bosh::Agent::Error: Request details: uri: https://vcap:b00tstrap@10.117.38.171:6868/agent payload: {"method":"ping","arguments":[],"reply_to":"bm-39d6fff6-b0dd-4751-ac96-b5e9e6b3686d"} user: vcap password: b00tstrap

wuj21 commented 11 years ago

Here is my micro_bosh.yml

name: micro01 #the same name as your folder name

XXX using micro01

logging: level: DEBUG network: ip: 10.117.38.171 #define ip for micro BOSH netmask: 255.255.255.0 gateway: 10.117.38.253 dns:

10.117.0.1
10.117.0.2 cloud_properties: name: VM Network #this should be the same as the “Network” of all your virtual machines
resources: persistent_disk: 16384 cloud_properties: ram: 4096 disk: 16384 cpu: 2 cloud: plugin: vsphere properties: agent: ntp:
- 10.112.64.1 vcenters:
  - host: 10.117.38.252 # ip address of vCenter user: Administrator #user of vCenter password: chorus123! datacenters:
    - name: bosh #name of the datacenter you created vm_folder: BOSH_VMS #folder for vms template_folder: BOSH_Templates #folder for templates disk_path: BOSH_Deployer #disk path where VMs are stored. datastore_pattern: boshdatastore #name of a datastore or some regular expression matching the datastore name. persistent_datastore_pattern: boshdatastore #name of a datastore or some regular expression matching the persistent datastore allow_mixed_datastores: true #true if two datastore patterns are the same clusters:
      - cfcluster #name of your cluster in your datacenter

apply_spec: agent: blobstore: address: 10.117.38.171 #change to micro bosh ip defined above nats: address: 10.117.38.171 #change to micro bosh ip defined above properties: {}

wuj21 commented 11 years ago

Here is gems I installed

jackwu@jack-ubuntu-bosh:~/deployments/micro01$ gem list

* LOCAL GEMS *

agent_client (1.5.0.pre.786) aws-sdk (1.8.5) bigdecimal (1.1.0) blobstore_client (1.5.0.pre.786) bosh_aws_cpi (1.5.0.pre.786) bosh_cli (1.5.0.pre.786) bosh_cli_plugin_micro (1.5.0.pre.786) bosh_common (1.5.0.pre.786) bosh_cpi (1.5.0.pre.786) bosh_openstack_cpi (1.5.0.pre.786) bosh_registry (1.5.0.pre.786) bosh_vcloud_cpi (1.5.0.pre.786) bosh_vsphere_cpi (1.5.0.pre.786) builder (3.1.4) bundler (1.3.5) daemons (1.1.9) eventmachine (1.0.3) excon (0.23.0) fog (1.12.1) formatador (0.2.4) gerrit-cli (0.0.1) highline (1.6.19) httpclient (2.2.4) io-console (0.3) json (1.5.5) json_pure (1.8.0) log4r (1.1.10) membrane (0.0.2) mime-types (1.23) minitar (0.5.4) minitest (2.5.1) multi_json (1.7.7) net-scp (1.1.1) net-ssh (2.6.7) net-ssh-gateway (1.1.0) netaddr (1.5.0) nokogiri (1.5.6) progressbar (0.9.2) psych (2.0.0) rack (1.5.2) rack-protection (1.5.0) rake (0.9.2.2) rdoc (3.9.5) rest-client (1.6.7) ruby-atmos-pure (1.0.5) ruby-hmac (0.4.0) ruby_vcloud_sdk (1.5.0.pre.786) ruby_vim_sdk (1.5.0.pre.786) rubygems-update (2.0.3) sequel (3.43.0) sinatra (1.4.3) sqlite3 (1.3.7) terminal-table (1.4.5) thin (1.5.1) tilt (1.4.1) uuidtools (2.1.4) yajl-ruby (1.1.0)

wuj21 commented 11 years ago

What's wrong?

drnic commented 11 years ago

Where are you getting your vSphere stemcell from? Is it from the s3 URL or (hopefully not at the moment) via "bosh public stemcells"?

Use the former. I'm not sure what the exact URL is; look for the OpenStack URLs and change it for vSphere and hopefully a stemcell will appear.

Dr Nic Williams Stark & Wayne LLC - the consultancy for Cloud Foundry http://starkandwayne.com +1 415 860 2185 twitter: drnic

On Thu, Jul 4, 2013 at 12:58 AM, Jack Wu notifications@github.com wrote:

What's wrong?

Reply to this email directly or view it on GitHub: https://github.com/cloudfoundry/bosh/issues/348#issuecomment-20463875

wuj21 commented 11 years ago

I got stemcells by

bosh public stemcells -all

and then

bosh download public stemcell micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz

wuj21 commented 11 years ago

drnic commented 11 years ago

Yep don't use those. They are very old and I guess don't work with latest bosh gems that you're using. Either try using old public bosh_cli (1.0.3) gem with that public stemcells; or figure out where the new pseudo-public vSphere stemcells are going.

Sorry I'm not being very helpful. It's been very confusing for everyone all year.

Dr Nic Williams Stark & Wayne LLC - the consultancy for Cloud Foundry http://starkandwayne.com +1 415 860 2185 twitter: drnic

On Thu, Jul 4, 2013 at 1:10 AM, Jack Wu notifications@github.com wrote:

jackwu@jack-ubuntu-bosh:~$ bosh public stemcells --full --all | grep micro | grep vsphere | micro-bosh-stemcell-vsphere-0.6.4.tgz | https://s3.amazonaws.com/blob.cfblob.com/stemcells/micro-bosh-stemcell-vsphere-0.6.4.tgz | vsphere, micro, stable | | micro-bosh-stemcell-vsphere-0.7.0.tgz | https://s3.amazonaws.com/blob.cfblob.com/stemcells/micro-bosh-stemcell-vsphere-0.7.0.tgz | vsphere, micro, test | | micro-bosh-stemcell-vsphere-0.8.1.tgz | https://s3.amazonaws.com/blob.cfblob.com/stemcells/micro-bosh-stemcell-vsphere-0.8.1.tgz | vsphere, micro, test | | micro-bosh-stemcell-vsphere-1.5.0.pre1.tgz | https://s3.amazonaws.com/blob.cfblob.com/stemcells/micro-bosh-stemcell-vsphere-1.5.0.pre1.tgz | vsphere, micro |

| micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz | https://s3.amazonaws.com/blob.cfblob.com/stemcells/micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz | vsphere, micro

Reply to this email directly or view it on GitHub: https://github.com/cloudfoundry/bosh/issues/348#issuecomment-20464346

xoebus commented 11 years ago

The latest micro bosh stemcell for vSphere can be downloaded here. It should work with the newer CLI and gems.

gabis commented 11 years ago

Please open a new issue if the stemcells @xoebus linked to do not work

cloudfoundry / bosh

Cannot install micro bosh in vSphere 5.1 #348

Stemcell info

Here is my micro_bosh.yml

XXX using micro01

What's wrong?

| micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz | https://s3.amazonaws.com/blob.cfblob.com/stemcells/micro-bosh-stemcell-vsphere-1.5.0.pre2.tgz | vsphere, micro