search

nathanmarz / storm-deploy

One click deploy for Storm clusters on AWS
Other
516 stars 148 forks source link

"com.jcraft.jsch.JSchException: fromBase64: invalid base64 data" while trying to deploy in EC2 #73

Open HIraolaE opened 9 years ago

HIraolaE commented 9 years ago

I tried this with the latest commits of the 3 branches of the project. All give me the same problems.

I am trying to deploy storm cluster in aws. The 4 virtual machines are created in my amazon account. However, when the message "com.jcraft.jsch.JSchException: fromBase64: invalid base64 data" apperas in the logs the execution stops and the 4 created virtual machines remain running in ec2. These machines have not storm installed.

I am running it OS X: 10.9.2 and java -version output looks like:

java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

My clusters.yaml looks like this

################################################################################
# CLUSTERS CONFIG FILE
################################################################################

nimbus.image: "us-east-1/ami-d726abbe"         #64-bit ubuntu
nimbus.hardware: "m1.large"

supervisor.count: 2
supervisor.image: "us-east-1/ami-d726abbe"         #64-bit ubuntu on eu-east-1
supervisor.hardware: "m1.large"
#supervisor.spot.price: 1.60

zookeeper.count: 1
zookeeper.image: "us-east-1/ami-d726abbe"         #64-bit ubuntu
zookeeper.hardware: "m1.large"

I am launching my cluster using the following command

lein deploy-storm --start --name clusterhodei --branch master --commit 0.9.0-rc2

The first line of the output looks like this:

DEBUG jclouds - Failed to find credentials annotation, using credentials supplier

It doesn't seem affected and the execution goes in its way. After a while these messages are logged:

DEBUG core - p-f-s server environment null
DEBUG core - p-f-s server environment null
DEBUG core - p-f-s server environment null
DEBUG core - p-f-s server environment null
DEBUG core - p-f-s server environment null
DEBUG core - p-f-s server environment null
DEBUG core - p-f-s server environment null
DEBUG core - p-f-s server environment null
INFO  core - parallel-apply-phase :pallet.phase/pre-configure for :zookeeper-mycluster
INFO  core - parallel-apply-phase-to-target :node  :pallet.phase/pre-configure for :zookeeper-mycluster with 1 nodes
INFO  core - parallel-apply-phase :pallet.phase/pre-configure for :supervisor-mycluster
INFO  core - parallel-apply-phase-to-target :node  :pallet.phase/pre-configure for :supervisor-mycluster with 2 nodes
INFO  core - parallel-apply-phase :pallet.phase/pre-configure for :nimbus-mycluster
INFO  core - parallel-apply-phase-to-target :node  :pallet.phase/pre-configure for :nimbus-mycluster with 1 nodes
DEBUG core - apply-phase-to-node: phase :pallet.phase/pre-configure group :zookeeper-mycluster target 174.129.175.130
DEBUG core - apply-phase-to-node: phase :pallet.phase/pre-configure group :supervisor-mycluster target 54.205.42.201
DEBUG core - apply-phase-to-node: phase :pallet.phase/pre-configure group :supervisor-mycluster target 54.163.157.22
DEBUG core - apply-phase-to-node: phase :pallet.phase/pre-configure group :nimbus-mycluster target 54.167.202.34
INFO  execute - execute-with-ssh on supervisor-mycluster "54.163.157.22"
INFO  execute - execute-with-ssh on nimbus-mycluster "54.167.202.34"
INFO  execute - execute-with-ssh on supervisor-mycluster "54.205.42.201"
INFO  execute - execute-with-ssh on zookeeper-mycluster "174.129.175.130"
ERROR logging - Exception in thread "main"
ERROR logging - java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.jcraft.jsch.JSchException: fromBase64: invalid base64 data (form-init8016119195565937948.clj:1)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415)
ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857)
ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820)
ERROR logging - at clojure.main$load_script.invoke(main.clj:221)
ERROR logging - at clojure.main$init_opt.invoke(main.clj:226)
ERROR logging - at clojure.main$initialize.invoke(main.clj:254)
ERROR logging - at clojure.main$null_opt.invoke(main.clj:279)
ERROR logging - at clojure.main$main.doInvoke(main.clj:354)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422)
ERROR logging - at clojure.lang.Var.invoke(Var.java:369)
ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165)
ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
ERROR logging - at clojure.main.main(main.java:37)
ERROR logging - Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.jcraft.jsch.JSchException: fromBase64: invalid base64 data
ERROR logging - at clojure.lang.LazySeq.sval(LazySeq.java:47)
ERROR logging - at clojure.lang.LazySeq.seq(LazySeq.java:56)
ERROR logging - at clojure.lang.RT.seq(RT.java:450)
ERROR logging - at clojure.core$seq.invoke(core.clj:122)
ERROR logging - at clojure.core$dorun.invoke(core.clj:2450)
ERROR logging - at clojure.core$doall.invoke(core.clj:2465)
ERROR logging - at pallet.core$parallel_lift.invoke(core.clj:757)
ERROR logging - at pallet.core$lift_phase$fn__5442.invoke(core.clj:779)
ERROR logging - at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
ERROR logging - at clojure.core$r.invoke(core.clj:797)
ERROR logging - at pallet.core$lift_phase.invoke(core.clj:781)
ERROR logging - at pallet.core$lift_nodes$fn__5445.invoke(core.clj:794)
ERROR logging - at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
ERROR logging - at clojure.core$r.invoke(core.clj:797)
ERROR logging - at pallet.core$lift_nodes.invoke(core.clj:798)
ERROR logging - at pallet.core$lift_STAR_.invoke(core.clj:1310)
ERROR logging - at pallet.core$converge_STAR_.invoke(core.clj:1335)
ERROR logging - at pallet.core$converge.doInvoke(core.clj:1506)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:440)
ERROR logging - at backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:69)
ERROR logging - at backtype.storm.provision$start_BANG_.invoke(provision.clj:91)
ERROR logging - at backtype.storm.provision$_main$fn__8907.invoke(provision.clj:144)
ERROR logging - at backtype.storm.provision$_main.doInvoke(provision.clj:130)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:552)
ERROR logging - at clojure.lang.Var.invoke(Var.java:390)
ERROR logging - at user$eval5.invoke(form-init8016119195565937948.clj:1)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424)
ERROR logging - ... 13 more
ERROR logging - Caused by: java.util.concurrent.ExecutionException: com.jcraft.jsch.JSchException: fromBase64: invalid base64 data
ERROR logging - at java.util.concurrent.FutureTask.report(FutureTask.java:122)
ERROR logging - at java.util.concurrent.FutureTask.get(FutureTask.java:188)
ERROR logging - at clojure.core$future_call$reify__5500.deref(core.clj:5399)
ERROR logging - at clojure.core$deref.invoke(core.clj:1765)
ERROR logging - at clojure.core$map$fn__3695.invoke(core.clj:2096)
ERROR logging - at clojure.lang.LazySeq.sval(LazySeq.java:42)
ERROR logging - ... 39 more
ERROR logging - Caused by: com.jcraft.jsch.JSchException: fromBase64: invalid base64 data
ERROR logging - at com.jcraft.jsch.Util.fromBase64(Util.java:63)
ERROR logging - at com.jcraft.jsch.KnownHosts.setKnownHosts(KnownHosts.java:222)
ERROR logging - at com.jcraft.jsch.KnownHosts.setKnownHosts(KnownHosts.java:60)
ERROR logging - at com.jcraft.jsch.JSch.setKnownHosts(JSch.java:303)
ERROR logging - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
ERROR logging - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
ERROR logging - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
ERROR logging - at java.lang.reflect.Method.invoke(Method.java:606)
ERROR logging - at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:90)
ERROR logging - at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
ERROR logging - at clj_ssh.ssh$ssh_agent.invoke(ssh.clj:103)
ERROR logging - at pallet.execute$default_agent$fn__4689.invoke(execute.clj:217)
ERROR logging - at clojure.lang.Atom.swap(Atom.java:37)
ERROR logging - at clojure.core$swap_BANG_.invoke(core.clj:1790)
ERROR logging - at pallet.execute$default_agent.invoke(execute.clj:213)
ERROR logging - at pallet.execute$execute_with_ssh$execute_with_ssh_fn__4858.invoke(execute.clj:604)
ERROR logging - at pallet.core$raise_on_error$fn__5216.invoke(core.clj:519)
ERROR logging - at pallet.core$middleware_handler$fn__5209.invoke(core.clj:495)
ERROR logging - at pallet.core$apply_phase_to_node.invoke(core.clj:656)
ERROR logging - at pallet.core$eval5379$fn__5380$iter__5381__5385$fn__5386$fn__5391.invoke(core.clj:721)
ERROR logging - at clojure.lang.AFn.call(AFn.java:18)
ERROR logging - at java.util.concurrent.FutureTask.run(FutureTask.java:262)
ERROR logging - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
ERROR logging - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
ERROR logging - at java.lang.Thread.run(Thread.java:744)
ERROR logging - Caused by: java.lang.ArrayIndexOutOfBoundsException: 437
ERROR logging - at com.jcraft.jsch.Util.fromBase64(Util.java:51)
ERROR logging - ... 24 more
vkuri commented 9 years ago

Facing the exact same issue on ubuntu with java version

root@storm:~/storm-deploy# java -version java version "1.7.0_75" OpenJDK Runtime Environment (IcedTea 2.5.4) (7u75-2.5.4-1~precise1) OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)

Also tried on master, install-0.9.1 and update-closure-jclouds branch as well. Stops at the exact same point always.