nathanmarz / storm-deploy

One click deploy for Storm clusters on AWS
Other
516 stars 148 forks source link

deploy to AWS error #58

Open venuktan opened 10 years ago

venuktan commented 10 years ago

cmd used to run:

sudo lein deploy-storm --start --name mycluster --branch master --commit 0.9.0-rc2

Starting cluster with storm branch master and commit 0.9.0-rc2 INFO provision - Provisioning nodes [nn=1, sn=2, zn=1] DEBUG core - pallet version: 0.7.3 INFO core - lift-nodes phases [:destroy-server], groups [] INFO core - lift-group-phase phases [:destroy-group], groups [] INFO core - lift-group-phase phases [:create-group], groups [:zookeeper-mycluster :supervisor-mycluster :nimbus-mycluster] INFO core - parallel-apply-phase :pallet.phase/pre-create-group for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :group :pallet.phase/pre-create-group for :zookeeper-mycluster INFO core - parallel-apply-phase :pallet.phase/pre-create-group for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :group :pallet.phase/pre-create-group for :supervisor-mycluster INFO core - parallel-apply-phase :pallet.phase/pre-create-group for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :group :pallet.phase/pre-create-group for :nimbus-mycluster INFO core - parallel-apply-phase :create-group for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :group :create-group for :zookeeper-mycluster INFO core - parallel-apply-phase :create-group for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :group :create-group for :supervisor-mycluster INFO core - parallel-apply-phase :create-group for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :group :create-group for :nimbus-mycluster INFO core - parallel-apply-phase :pallet.phase/post-create-group for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :group :pallet.phase/post-create-group for :zookeeper-mycluster INFO core - parallel-apply-phase :pallet.phase/post-create-group for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :group :pallet.phase/post-create-group for :supervisor-mycluster INFO core - parallel-apply-phase :pallet.phase/post-create-group for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :group :pallet.phase/post-create-group for :nimbus-mycluster INFO core - adjust-server-count :zookeeper-mycluster INFO core - adjust-server-count :supervisor-mycluster INFO core - adjust-server-count :nimbus-mycluster INFO core - Starting 1 nodes for :nimbus-mycluster, os-family null INFO core - Starting 1 nodes for :zookeeper-mycluster, os-family null INFO core - Starting 2 nodes for :supervisor-mycluster, os-family null ERROR futures - Adjust node count exception: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil java.lang.IllegalArgumentException: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:471) at pallet.compute$eval1063$fn__1092$G__1046__1099.invoke(compute.clj:49) at pallet.core$create_nodes$fn__5444.invoke(core.clj:837) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:542) at clojure.core$update_in.doInvoke(core.clj:4959) at clojure.lang.RestFn.invoke(RestFn.java:446) at pallet.core$create_nodes.invoke(core.clj:836) at pallet.core$adjust_node_count.invoke(core.clj:912) at pallet.core$parallel_adjust_node_counts$p_a_n_c_future__5506$fn__5507.invoke(core.clj:1003) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) DEBUG futures - Adjust node count exception java.lang.IllegalArgumentException: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:471) at pallet.compute$eval1063$fn__1092$G__1046__1099.invoke(compute.clj:49) at pallet.core$create_nodes$fn__5444.invoke(core.clj:837) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:542) at clojure.core$update_in.doInvoke(core.clj:4959) at clojure.lang.RestFn.invoke(RestFn.java:446) at pallet.core$create_nodes.invoke(core.clj:836) at pallet.core$adjust_node_count.invoke(core.clj:912) at pallet.core$parallel_adjust_node_counts$p_a_n_c_future__5506$fn__5507.invoke(core.clj:1003) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) ERROR futures - Adjust node count exception: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil java.lang.IllegalArgumentException: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:471) at pallet.compute$eval1063$fn__1092$G__1046__1099.invoke(compute.clj:49) at pallet.core$create_nodes$fn__5444.invoke(core.clj:837) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:542) at clojure.core$update_in.doInvoke(core.clj:4959) at clojure.lang.RestFn.invoke(RestFn.java:446) at pallet.core$create_nodes.invoke(core.clj:836) at pallet.core$adjust_node_count.invoke(core.clj:912) at pallet.core$parallel_adjust_node_counts$p_a_n_c_future__5506$fn__5507.invoke(core.clj:1003) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) DEBUG futures - Adjust node count exception java.lang.IllegalArgumentException: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:471) at pallet.compute$eval1063$fn__1092$G__1046__1099.invoke(compute.clj:49) at pallet.core$create_nodes$fn__5444.invoke(core.clj:837) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:542) at clojure.core$update_in.doInvoke(core.clj:4959) at clojure.lang.RestFn.invoke(RestFn.java:446) at pallet.core$create_nodes.invoke(core.clj:836) at pallet.core$adjust_node_count.invoke(core.clj:912) at pallet.core$parallel_adjust_node_counts$p_a_n_c_future__5506$fn__5507.invoke(core.clj:1003) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) ERROR futures - Adjust node count exception: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil java.lang.IllegalArgumentException: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:471) at pallet.compute$eval1063$fn__1092$G__1046__1099.invoke(compute.clj:49) at pallet.core$create_nodes$fn__5444.invoke(core.clj:837) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:542) at clojure.core$update_in.doInvoke(core.clj:4959) at clojure.lang.RestFn.invoke(RestFn.java:446) at pallet.core$create_nodes.invoke(core.clj:836) at pallet.core$adjust_node_count.invoke(core.clj:912) at pallet.core$parallel_adjust_node_counts$p_a_n_c_future__5506$fn__5507.invoke(core.clj:1003) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) DEBUG futures - Adjust node count exception java.lang.IllegalArgumentException: No implementation of method: :ensure-os-family of protocol: #'pallet.compute/ComputeService found for class: nil at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:471) at pallet.compute$eval1063$fn__1092$G__1046__1099.invoke(compute.clj:49) at pallet.core$create_nodes$fn__5444.invoke(core.clj:837) at clojure.lang.AFn.applyToHelper(AFn.java:163) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:542) at clojure.core$update_in.doInvoke(core.clj:4959) at clojure.lang.RestFn.invoke(RestFn.java:446) at pallet.core$create_nodes.invoke(core.clj:836) at pallet.core$adjust_node_count.invoke(core.clj:912) at pallet.core$parallel_adjust_node_counts$p_a_n_c_future__5506$fn__5507.invoke(core.clj:1003) at clojure.lang.AFn.call(AFn.java:18) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) DEBUG core - pallet version: 0.7.3 INFO core - lift-nodes phases [:settings :configure], groups [:zookeeper-mycluster :supervisor-mycluster :nimbus-mycluster] INFO core - parallel-apply-phase :pallet.phase/pre-settings for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/pre-settings for :zookeeper-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/pre-settings for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/pre-settings for :supervisor-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/pre-settings for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/pre-settings for :nimbus-mycluster with 0 nodes INFO core - parallel-apply-phase :settings for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :node :settings for :zookeeper-mycluster with 0 nodes INFO core - parallel-apply-phase :settings for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :node :settings for :supervisor-mycluster with 0 nodes INFO core - parallel-apply-phase :settings for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :node :settings for :nimbus-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/post-settings for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/post-settings for :zookeeper-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/post-settings for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/post-settings for :supervisor-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/post-settings for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/post-settings for :nimbus-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/pre-configure for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/pre-configure for :zookeeper-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/pre-configure for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/pre-configure for :supervisor-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/pre-configure for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/pre-configure for :nimbus-mycluster with 0 nodes INFO core - parallel-apply-phase :configure for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :node :configure for :zookeeper-mycluster with 0 nodes INFO core - parallel-apply-phase :configure for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :node :configure for :supervisor-mycluster with 0 nodes INFO core - parallel-apply-phase :configure for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :node :configure for :nimbus-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/post-configure for :zookeeper-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/post-configure for :zookeeper-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/post-configure for :supervisor-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/post-configure for :supervisor-mycluster with 0 nodes INFO core - parallel-apply-phase :pallet.phase/post-configure for :nimbus-mycluster INFO core - parallel-apply-phase-to-target :node :pallet.phase/post-configure for :nimbus-mycluster with 0 nodes DEBUG provision - Finished converge ERROR logging - Exception in thread "main" ERROR logging - java.lang.NullPointerException (form-init3395142206682442835.clj:1) ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440) ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415) ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857) ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820) ERROR logging - at clojure.main$load_script.invoke(main.clj:221) ERROR logging - at clojure.main$init_opt.invoke(main.clj:226) ERROR logging - at clojure.main$initialize.invoke(main.clj:254) ERROR logging - at clojure.main$null_opt.invoke(main.clj:279) ERROR logging - at clojure.main$main.doInvoke(main.clj:354) ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422) ERROR logging - at clojure.lang.Var.invoke(Var.java:369) ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165) ERROR logging - at clojure.lang.Var.applyTo(Var.java:482) ERROR logging - at clojure.main.main(main.java:37) ERROR logging - Caused by: java.lang.NullPointerException ERROR logging - at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:263) ERROR logging - at backtype.storm.security$authorize_group.invoke(security.clj:111) ERROR logging - at backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:77) ERROR logging - at backtype.storm.provision$start_BANG_.invoke(provision.clj:91) ERROR logging - at backtype.storm.provision$_main$fn__8422.invoke(provision.clj:144) ERROR logging - at backtype.storm.provision$_main.doInvoke(provision.clj:130) ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:552) ERROR logging - at clojure.lang.Var.invoke(Var.java:390) ERROR logging - at user$eval5$fn__7.invoke(form-init3395142206682442835.clj:1) ERROR logging - at user$eval5.invoke(form-init3395142206682442835.clj:1) ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424) ERROR logging - ... 13 more

venuktan commented 10 years ago

When I do not use sudo as below I get lein deploy-storm --start --name test --branch 0.8.2

Could not transfer artifact org.cloudhoist:pallet-pom:pom:0.7.3 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:pallet-jclouds:pom:1.5.1 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:java:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:git:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:ssh-key:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:automated-admin-user:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:iptables:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:maven:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:zookeeper:pom:0.5.1 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:nagios-config:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:crontab:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated This could be due to a typo in :dependencies or network issues. If you are behind a proxy, try setting the 'http_proxy' environment variable.

tbatchelli commented 10 years ago

In the latter case it seems like either you are having proxy issues, the maven repo is unavailable, or there is some misconfiguration in project.clj. 

What happens if you do ‘lein deps'?

tbatchelli Sent with Airmail

On March 19, 2014 at 4:23:51 PM, Venu K Tangirala (notifications@github.com) wrote:

When I do not use sudo as below I get lein deploy-storm --start --name test --branch 0.8.2

lein deploy-storm --start --name mycluster --branch master --commit 0.9.0-rc2 Could not transfer artifact org.cloudhoist:pallet-pom:pom:0.7.3 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:pallet-jclouds:pom:1.5.1 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:java:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:git:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:ssh-key:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:automated-admin-user:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:iptables:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:maven:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:zookeeper:pom:0.5.1 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:nagios-config:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated Could not transfer artifact org.cloudhoist:crontab:pom:0.5.0 from/to jclouds-snapshot (https://oss.sonatype.org/content/repositories/snapshots): peer not authenticated This could be due to a typo in :dependencies or network issues. If you are behind a proxy, try setting the 'http_proxy' environment variable.


Reply to this email directly or view it on GitHub: https://github.com/nathanmarz/storm-deploy/issues/58#issuecomment-38119784

stephenHartzell commented 10 years ago

I am getting the exact same error as venuktan when I run sudo lein deploy-storm --start --name test.

I am actually trying to run storm-deploy from an AWS instance because of some proxy issues on my work machine. I tried following the tutorial exactly on a bran new centOS AWS machine and got this exact same error. Any thoughts?

rsnider19 commented 8 years ago

I'm getting same error when running in an aws bitnami ubuntu instance

baannee commented 7 years ago

Same issue for me on 2 different regions, with 2-3 different types of AMIs. Any update please? Thanks in advance!