Issue while bosh-init on AWS

[Senthilannaswamy]

I'm trying to bosh-init, by following below reference.

http://bosh.io/docs/init-aws.html

While running "bosh-init deploy ./bosh.yml", getting error like, (please note, have tried Uncommenting "properties:nats:password", still same issue). Please suggest.

Command 'deploy' failed:
  Installing CPI:
    Rendering and uploading Jobs:
      Rendering job templates for installation:
        Rendering templates for job 'aws_cpi/2a34fab07ff9a36e43ede22e227c02918a5ccdef':
          Rendering template src: cpi.json.erb, dst: config/cpi.json:
            Rendering template src: /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/bosh-init-release942259122/extracted_jobs/aws_cpi/templates/cpi.json.erb, dst: /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/rendered-jobs471613702/config/cpi.json:
              Running ruby to render templates:
                Running command: 'ruby /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-context.json /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/bosh-init-release942259122/extracted_jobs/aws_cpi/templates/cpi.json.erb /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/rendered-jobs471613702/config/cpi.json', stdout: '', stderr: '/home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb:189:in `rescue in render': Error filling in template '/home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/bosh-init-release942259122/extracted_jobs/aws_cpi/templates/cpi.json.erb' for aws_cpi/0 (line 92: #<TemplateEvaluationContext::UnknownProperty: Can't find property 'nats.password'>) (RuntimeError)
        from /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb:175:in `render'
        from /home/ec2-user/.bosh_init/installations/e10f34eb-3e4d-4377-5320-132e9ec3054a/tmp/erb-renderer232054824/erb-render.rb:200:in `<main>'

Deployment manifest file was changed only for following properties,

1. ELASTIC-IP,
2. SUBNET-ID,
3. REGION (e.g. us-east-1),
4. AVAILABILITY-ZONE (e.g. us-east-1a),
5. ACCESS-KEY-ID,
6. SECRET-ACCESS-KEY .

---
name: bosh

releases:
- name: bosh
  url: https://bosh.io/d/github.com/cloudfoundry/bosh?v=261.2
  sha1: d4635b4b82b0dc5fd083b83eb7e7405832f6654b
- name: bosh-aws-cpi
  url: https://bosh.io/d/github.com/cloudfoundry-incubator/bosh-aws-cpi-release?v=62
  sha1: f36967927ceae09e5663a41fdda199edfe649dc6

resource_pools:
- name: vms
  network: private
  stemcell:
    url: https://bosh.io/d/stemcells/bosh-aws-xen-hvm-ubuntu-trusty-go_agent?v=3363.9
    sha1: c5a61384178efaade80210de7ebf994d70e6ca25
  cloud_properties:
    instance_type: m3.xlarge
    ephemeral_disk: {size: 25_000, type: gp2}
    availability_zone: us-east-1b # <--- Replace with Availability Zone

disk_pools:
- name: disks
  disk_size: 20_000
  cloud_properties: {type: gp2}

networks:
- name: private
  type: manual
  subnets:
  - range: 10.11.0.0/24
    gateway: 10.11.0.1
    dns: [10.11.0.2]
    cloud_properties: {subnet: subnet-f07640dd} # <--- Replace with Subnet ID
- name: public
  type: vip

jobs:
- name: bosh
  instances: 1

  templates:
  - {name: nats, release: bosh}
  - {name: postgres, release: bosh}
  - {name: blobstore, release: bosh}
  - {name: director, release: bosh}
  - {name: health_monitor, release: bosh}
  - {name: registry, release: bosh}
  - {name: aws_cpi, release: bosh-aws-cpi}

  resource_pool: vms
  persistent_disk_pool: disks

  networks:
  - name: private
    static_ips: [10.11.0.6]
    default: [dns, gateway]
  - name: public
    static_ips: [34.192.209.204] # <--- Replace with Elastic IP

  properties:
    nats:
      address: 127.0.0.1
      user: nats
      # password: nats-password # <--- Uncomment & change

    postgres: &db
      listen_address: 127.0.0.1
      host: 127.0.0.1
      user: postgres
      # password: postgres-password # <--- Uncomment & change
      database: bosh
      adapter: postgres

    registry:
      address: 10.11.0.6
      host: 10.11.0.6
      db: *db
      http:
        user: admin
        # password: admin # <--- Uncomment & change
        port: 25777
      username: admin
      # password: admin # <--- Uncomment & change
      port: 25777

    blobstore:
      address: 10.11.0.6
      port: 25250
      provider: dav
      director:
        user: director
        # password: director-password # <--- Uncomment & change
      agent:
        user: agent
        # password: agent-password # <--- Uncomment & change

    director:
      address: 127.0.0.1
      name: my-bosh
      db: *db
      cpi_job: aws_cpi
      max_threads: 10
      user_management:
        provider: local
        local:
          users:
          # - {name: admin, password: admin} # <--- Uncomment & change
          # - {name: hm, password: hm-password} # <--- Uncomment & change

    hm:
      director_account:
        user: hm
        # password: hm-password # <--- Uncomment & change
      resurrector_enabled: true

    aws: &aws
      access_key_id: {{redacted}} # <--- Replace with AWS Access Key ID
      secret_access_key: {{redacted}} # <--- Replace with AWS Secret Key
      default_key_name: bosh
      default_security_groups: [offshore-bosh-sg]
      region: us-east-1  # <--- Replace with Region

    # agent: {mbus: "nats://nats:nats-password@10.0.0.6:4222"} # <--- Uncomment & change

    ntp: &ntp [0.pool.ntp.org, 1.pool.ntp.org]

cloud_provider:
  template: {name: aws_cpi, release: bosh-aws-cpi}

  ssh_tunnel:
    host: 34.192.209.204 # <--- Replace with your Elastic IP address
    port: 22
    user: vcap
    private_key: ./bosh.pem # Path relative to this manifest file

  # mbus: "https://mbus:mbus-password@34.192.209.204:6868" # <--- Uncomment & change

  properties:
    aws: *aws
    # agent: {mbus: "https://mbus:mbus-password@0.0.0.0:6868"} # <--- Uncomment & change
    blobstore: {provider: local, path: /var/vcap/micro_bosh/data/cache}
    ntp: *ntp

[dpb587-pivotal]

Please take note of the "Uncomment & change" sections and make sure you have uncommented them and provided non-default passwords for them - they're required for deployment.

Also, please rotate your access key and secret access key ending in LIQA - it should have been redacted and has now been emailed out to watchers of this repo.

[jacko1118]

dear all.

I got the same error and stuck there even I have uncomment and put a password.

Could you pls help. Thanks.

Command 'deploy' failed: Installing CPI: Rendering and uploading Jobs: Rendering job templates for installation: Rendering templates for job 'vsphere_cpi/677eb08a995a5bfde2ad0ed9c4ec8c0914272c47': Rendering template src: cpi.json.erb, dst: config/cpi.json: Rendering template src: /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/bosh-init-release722782197/extracted_jobs/vsphere_cpi/templates/cpi.json.erb, dst: /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/rendered-jobs717219588/config/cpi.json: Running ruby to render templates: Running command: 'ruby /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-context.json /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/bosh-init-release722782197/extracted_jobs/vsphere_cpi/templates/cpi.json.erb /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/rendered-jobs717219588/config/cpi.json', stdout: '', stderr: '/root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb:189:in rescue in render': Error filling in template '/root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/bosh-init-release722782197/extracted_jobs/vsphere_cpi/templates/cpi.json.erb' for vsphere_cpi/0 (line 112: #<TemplateEvaluationContext::UnknownProperty: Can't find property 'nats.password'>) (RuntimeError) from /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb:175:inrender' from /root/.bosh_init/installations/afc81326-c887-4809-5b11-dd90070b0422/tmp/erb-renderer631514685/erb-render.rb:200:in `

' ': exit status 1 root@ubuntu6:~/my-bosh#

[Senthilannaswamy]

HI Jacko1118,

Can you share the bosh.yml?

Regards Senthil.

[jacko1118]

---

name: bosh

releases:

• name: bosh url: https://bosh.io/d/github.com/cloudfoundry/bosh?v=261.4 sha1: 4da9cedbcc8fbf11378ef439fb89de08300ad091
• name: bosh-vsphere-cpi url: https://bosh.io/d/github.com/cloudfoundry-incubator/bosh-vsphere-cpi-release?v=39 sha1: 2b68e45b553c3867fc4f7021a723eae7ec3c23fd

resource_pools:

• name: vms network: private stemcell: url: https://bosh.io/d/stemcells/bosh-vsphere-esxi-ubuntu-trusty-go_agent?v=3363.15 sha1: d5571cd8e13d1daca99dc821e4fb751f4cdd42f8 cloud_properties: cpu: 2 ram: 4_096 disk: 20_000 env: bosh:

  c1oudc0w is a default password for vcap user

  password: "$6$4gDD3aV0rdqlrKC$2axHCxGKIObs6tAmMTqYCspcdvQXh3JJcvWOY2WGb4SrdXtnCyNaWlrf3WEqvYR2MYizEGp3kMmbpwBC6jsHt0"

disk_pools:

• name: disks disk_size: 20_000

networks:

• name: private type: manual subnets:
  • range: 10.1.1.140/24 gateway: 10.1.1.100 dns: [10.1.1.100] cloud_properties: {name: VM-Network } # <--- Replace with Network name

jobs:

• name: bosh instances: 1

  templates:

  • {name: nats, release: bosh}
  • {name: postgres, release: bosh}
  • {name: blobstore, release: bosh}
  • {name: director, release: bosh}
  • {name: health_monitor, release: bosh}
  • {name: vsphere_cpi, release: bosh-vsphere-cpi}

  resource_pool: vms persistent_disk_pool: disks

  networks:

  • {name: private, static_ips: [10.1.1.140]}

  properties: nats: address: 10.1.1.141 user: nats password: "Test1111q" # <--- Uncomment & change

  postgres: &db listen_address: 10.1.1.142 host: 127.0.0.1 user: PoStGrEs password: pOsTgReS # <--- Uncomment & change database: bosh adapter: postgres

  blobstore: address: 10.1.1.145 port: 25250 provider: dav director: user: DiReCtOr password: dIrEcToR # <--- Uncomment & change agent: user: AgEnT password: aGeNt # <--- Uncomment & change

  director: address: 10.1.1.144 name: my-bosh db: *db cpi_job: vsphere_cpi user_management: provider: local local: users:

  • {name: admin, password: admin} # <--- Uncomment & change
  • {name: AdMiN, password: pAsSwOrD} # <--- Uncomment & change

  hm: director_account: user: AdMiN password: pAsSwOrD # <--- Uncomment & change resurrector_enabled: true intervals: agent_timeout: 180

  vcenter: &vcenter # <--- Replace values below address: 10.1.1.120 user: administrator@vsphere.local password: Hugo@8888 datacenters:

  • name: sodec_cf vm_folder: vm_folder template_folder: template_folder_bosh datastore_pattern: datastore1 persistent_datastore_pattern: datastore1 disk_path: my-bosh-disks clusters: [test_cluster]

    agent: {mbus: "nats://nats:test11@10.0.0.6:4222"} # <--- Uncomment & change

  ntp: &ntp [0.pool.ntp.org, 1.pool.ntp.org]

cloud_provider: template: {name: vsphere_cpi, release: bosh-vsphere-cpi}

mbus: "https://mbus:mbus-password@10.0.0.6:6868" # <--- Uncomment & change

properties: vcenter: vcenter agent: {mbus: "https://mbus:mbus-password@0.0.0.0:6868"} # <--- Uncomment & change blobstore: {provider: local, path: /var/vcap/micro_bosh/data/cache} ntp: ntp

[jacko1118]

dear Senthil,

I got the same error if let the password field uncomment.

If I change the field , I got below error:

root@ubuntu6:~/my-bosh# bosh-init deploy ./bosh.yml Deployment manifest: '/root/my-bosh/bosh.yml' Deployment state: '/root/my-bosh/bosh-state.json'

Started validating Failed validating (00:00:00)

Command 'deploy' failed: Parsing release set manifest '/root/my-bosh/bosh.yml': Unmarshalling release set manifest: yaml: line 61: mapping values are not allowed in this context root@ubuntu6:~/my-bosh#

[jacko1118]

templates:

• {name: nats, release: bosh}
• {name: postgres, release: bosh}
• {name: blobstore, release: bosh}
• {name: director, release: bosh}
• {name: health_monitor, release: bosh}

• {name: vsphere_cpi, release: bosh-vsphere-cpi}

  resource_pool: vms persistent_disk_pool: disks

  networks:

• {name: private, static_ips: [10.1.1.140]}

  properties: nats: address: 10.1.1.141 user: natsjacko password: Test1111q # <--- Uncomment & change

                                                                              61,1          35%

Command 'deploy' failed: Parsing release set manifest '/root/my-bosh/bosh.yml': Unmarshalling release set manifest: yaml: line 61: mapping values are not allowed in this context

you can see the log detect my error in line 61 which is the line for nats user. I am stucking here. could you pls help?

[Senthilannaswamy]

HI jacko1118,

Its looks like intent issue, can you attach bosh.yml as file (you need to change the extension of the file). -Senthil

[jacko1118]

hi Senthil, finally I success to deploy the bosh director with below bosh.yml. the format of the bosh.yml is very strictly to control, even more than one space I would get the error

thanks.

[drnic]

If it helps, here's a blog post on deploying bosh to AWS using bosh create-env + new bosh-deployment repo (which are replacing bosh-init) https://www.starkandwayne.com/blog/bootstrap-bosh-2-0-on-aws/

[dpb587-pivotal]

Senthil, thanks for helping out others here! Were you able to resolve your original issue that this can be closed?

[Senthilannaswamy]

HI There,

Yes, you can consider as closed.

Senthil.