Closed drnic closed 9 years ago
The manifest I used was https://gist.github.com/drnic/f953407db9fcbb24a050
I'm investigating.
BTW, doing nothing but making bosh-micro deploy
errors go away, the following is the minimum manifest I got away with:
---
name: redis-from-scratch
resource_pools:
- name: default
network: default
cloud_properties:
instance_type: m3.medium
networks:
- name: default
type: manual
cloud_properties:
subnet: subnet-37086b40
range: 10.10.0.0/24
reserved: [10.10.0.1-10.10.0.3]
static: [10.10.0.4]
cloud_provider:
template:
name: cpi
release: bosh-aws-cpi
I didn't test smaller networks
and resource_pools
sections; I copied them from the gist above.
Eventual after CPI compilation, it continues failing with:
Started installing CPI
Compiling package 'ruby_aws_cpi/052a28b8976e6d9ad14d3eaec6d3dd237973d800'... Finished (00:01:18)
Compiling package 'bosh_aws_cpi/deabbf731a4fedc9285324d85af6456cfa74c10c'... Finished (00:00:34)
Rendering job templates... Failed (00:00:00)
Failed installing CPI (00:01:53)
Command 'deploy' failed:
Building installation state:
Rendering job templates for installation:
Rendering templates for job 'cpi/ca1bbc783f6b12eb1d066ac4b54c75fb8351465e':
Rendering template src: cpi.yml.erb, dst: config/cpi.yml:
Rendering template src: /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/bosh-micro-release459033935/extracted_jobs/cpi/templates/cpi.yml.erb, dst: /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/re$dered-jobs298615990/config/cpi.yml:
Running ruby to render templates:
Running command: 'ruby /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/erb-renderer013178712/erb-render.rb /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/erb-renderer013178712/erb-context.json /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/bosh-micro-release459033935/extracted_jobs/cpi/templates/cpi.yml.erb /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/rendered-jobs298615990/config/cpi.yml', st$out: '', stderr: '/var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/erb-renderer013178712/erb-render.rb:180:in `rescue in render': Error filling in template '/var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/bos$-micro-release459033935/extracted_jobs/cpi/templates/cpi.yml.erb' for cpi/0 (line 5: #<TemplateEvaluationContext::UnknownProperty: Can't find property 'aws.access_key_id'>) (RuntimeError)
from /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/erb-renderer013178712/erb-render.rb:166:in `render'
from /var/folders/3x/fch8r6z97ljct1tfzk7kk34w0000gn/T/erb-renderer013178712/erb-render.rb:191:in `<main>'
But now the error message isn't as pretty as it was prior to compiling the CPI.
---
name: redis-from-scratch
resource_pools:
- name: default
network: default
cloud_properties:
instance_type: m3.medium
networks:
- name: default
type: manual
cloud_properties:
subnet: subnet-37086b40
range: 10.10.0.0/24
reserved: [10.10.0.1-10.10.0.3]
static: [10.10.0.4]
cloud_provider:
template:
name: cpi
release: bosh-aws-cpi
properties:
aws:
access_key_id: XXX
secret_access_key: YYY
default_key_name: sw-bosh-us-east-1
default_security_groups: [bastion]
region: us-east-1
ec2_private_key: ~/.ssh/sw-bosh-us-east-1.pem
registry:
username: admin
password: admin
port: 6901
host: localhost
blobstore:
provider: local
path: /var/vcap/micro_bosh/data/cache
nats:
address: 127.0.0.1
password: nats
Was the minimum before it would try to create a stemcell.
Then it fails with:
Command 'deploy' failed:
Deploying Microbosh:
There must only be one job, found 0
I add the jobs section and then it panics:
jobs:
- name: redis
instances: 1
templates:
- name: redis
release: redis
networks:
- name: default
static_ips:
- 10.10.0.4
properties:
properties:
redis:
address: "127.0.0.1"
password: "redis"
port: 25255
properties:
properties:
Was the issue here. But golang panicing cannot be the UX for bad YAML. Hopefully there is something we can do to aide fools like me who write back YAML.
Perhaps a simple pre-marshaling effort to check "can this YAML be parsed as YAML?" "There's a YAML error on line XYZ"
Going to keep using this thread as a log of one person's efforts to build up a manifest.
Was getting this error:
Command 'deploy' failed:
Deploying Microbosh:
Creating instance 'redis/0':
Creating VM:
Creating vm with stemcell cid 'ami-b88da7d0 light':
External CPI command for method 'create_vm' returned an error: CmdError{"type":"Unknown","message":"Connection refused - connect(2) (http://localhost:6901)","ok_to_retry":false}
This went away when I added the seemingly redundant cloud_provider.registrar
:
cloud_provider:
template:
name: cpi
release: bosh-aws-cpi
ssh_tunnel:
host: 10.10.0.4
port: 22
user: vcap
private_key: /home/ubuntu/.ssh/sw-bosh-us-east-1.pem
registry:
username: admin
password: admin
port: 6901
host: localhost
properties:
aws:
access_key_id: XXX
secret_access_key: YYY
default_key_name: sw-bosh-us-east-1
default_security_groups: [redis-micro-default-vpc]
region: us-east-1
ec2_private_key: ~/.ssh/sw-bosh-us-east-1.pem
registry:
username: admin
password: admin
port: 6901
host: localhost
blobstore:
provider: local
path: /var/vcap/micro_bosh/data/cache
nats:
address: 127.0.0.1
password: nats
Seems strange to have to enter all this - most of it seems to be defaults. Perhaps they can be optional - would make the manifest much smaller.
Needed to fix path to key in cloud_provider. ssh_tunnel. private_key
since I was on a different machine.
Tried ~/.ssh/sw-bosh-us-east-1.pem
(like in ec2_private_key
) but get:
Command 'deploy' failed:
Deploying Microbosh:
Creating instance 'redis/0':
Waiting until instance is ready:
Starting SSH tunnel:
Reading private key file '~/.ssh/sw-bosh-us-east-1.pem':
open ~/.ssh/sw-bosh-us-east-1.pem: no such file or directory
This issue is already raised in https://github.com/cloudfoundry/bosh-micro-cli/issues/4 but no response from anyone
With the ssh key fixed, the agent wasn't coming up. SSHing into the VM and looking at /var/vcap/bosh/log/current
I see:
2015-04-04_04:24:26.38129 [settingsService] 2015/04/04 04:24:26 DEBUG - Loading settings from fetcher
2015-04-04_04:24:26.38186 [registryProvider] 2015/04/04 04:24:26 DEBUG - Using http registry at http://admin:admin@localhost:6901
2015-04-04_04:24:26.39146 [settingsService] 2015/04/04 04:24:26 ERROR - Failed loading settings via fetcher: Getting settings from url: Get http://admin:admin@localhost:6901/instances/i-f50bc508/settings: dial tcp 127.0.0.1:6901: connection refused
2015-04-04_04:24:26.39158 [File System] 2015/04/04 04:24:26 DEBUG - Reading file /var/vcap/bosh/settings.json
2015-04-04_04:24:26.39169 [settingsService] 2015/04/04 04:24:26 ERROR - Failed reading settings from file Opening file /var/vcap/bosh/settings.json: open /var/vcap/bosh/settings.json: no such file or directory
2015-04-04_04:24:26.39180 [main] 2015/04/04 04:24:26 ERROR - App setup Running bootstrap: Fetching settings: Invoking settings fetcher: Getting settings from url: Get http://admin:admin@localhost:6901/instances/i-f50bc508/settings: dial tcp 127.0.0.1:6901: connection refused
At this time, on the host machine there is no other related process running (no registry process) other than bosh-micro deploy
.
I'll keep trying tonight if anyone / @cppforlife is around and can suggest something.
I am running this on my OS X machine into us-east-1. I've added an elastic IP in addition to the static IP above.
Did you resolve this problem?
FYI, I raised the issue of the stack trace and its inscrutibility with @fraenkel in this PR: https://github.com/fraenkel/candiedyaml/issues/5
He has since added commits to https://github.com/cloudfoundry-incubator/candiedyaml to improve failure messages when unmarshalling. The dependency will need to be bumped in bosh-init to take advantage of this.
We have switched to using go-yaml library which provides better error messages for invalid YAML.
I tried again today to deploy a simple redis/redis micro deployment and when I first try I get golang panics unmarshalling the manifest:
In my 3 years of BOSH, no one ever gets a manifest right initially; and then they get them wrong a lot of the time later. I don't think panicing is a good UX :)
Any ideas/plans on how we will solve this?