search

riptano / ComboAMI

The AMI takes a set of input parameters via the EC2 user-data to install, RAID, ring, and launch a DataStax Enterprise/Community cluster.
69 stars 59 forks source link

Unable to start a cluster using DataStax Auto-Clustering AMI 2.6-1404-hvm - ami-35ac725e #84

Closed cooniur closed 9 years ago

cooniur commented 9 years ago

Stuck at Waiting for cluster to boot.... The server doesn't make any progress after more than one hour.

Cluster configuration is --clustername my_cluster --totalnodes 1 --version community --opscenter yes.

In System Log of the instance, I can see lines of [INFO] Waiting for cloud-init to finish....

It happens on both instance type m3.medium and m3.large. Both of the instances have access to the internet. (i.e. I'm able to curl http://www.google.com and get response) One is inside a private subnet using NAT to get access to the internet. The other has a public IP.

I don't know why it stuck there. Did I do anything wrong? Thanks!

mlococo commented 9 years ago

Investigating.

cooniur commented 9 years ago

@mlococo To give you some updates, I tried 2.5 (datastax_clustering_ami_2.5.1_hvm.manifest.xml (ami-ada2b6c4)) and it works. I'm able to get Cassandra cluster started and enter the OpsCenter.

Thanks!

mlococo commented 9 years ago

I added the cloud-init wait shortly before release after testing it during packer builds, where it worked well. Unfortunately, the AMI startup scripts run as rc-scripts, and cloud-init itself waits for them to complete before finishing, creating a cyclic dependency.

2.6.1 has been released which fixes the issue. I've de-registered all the 2.6.0 AMI's since they don't actually work.

cooniur commented 9 years ago

@mlococo Thanks for the fix!

mlococo commented 9 years ago

No problem, thanks for the clear bug report.