Storm-deploy with Storm >= 0.9.0

[ssolbak]

I spent about the entire day yesterday trying to do the basic storm deploy commands. I've tried on ubuntu, mac and a aws box. The deploy works for storm 0.8.2 but our topoligies are written with storm 0.9.0.1.

The error I'm getting is (I think its the first one)

-------begin error 2014-02-09 18:57:39,833 INFO pallet.execute Output: 54.184.12.237 Already up-to-date. bash: bin/build_release.sh: No such file or directory Build storm failed logout

2014-02-09 18:57:39,842 INFO pallet.execute Output: 54.184.41.36 checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking how to run the C++ preprocessor... g++ -E

2014-02-09 18:57:39,936 ERROR pallet.execute Exit status : 1 -------end error

The commands Im using are (Ive tried numerous branches)

lein deploy-storm --start --name test lein deploy-storm --start --name test --branch 0.9.0.1 lein deploy-storm --start --name test --branch 0.9.0

My config is as follows. I was testing with t1.micros but now on m1.smalls.

clusters.yaml nimbus.image: "us-west-2/ami-84d7b7b4" nimbus.hardware: "m1.small"

supervisor.count: 2 supervisor.image: "us-west-2/ami-84d7b7b4" supervisor.hardware: "m1.small"

supervisor.spot.price: 1.60

zookeeper.count: 1 zookeeper.image: "us-west-2/ami-84d7b7b4" zookeeper.hardware: "m1.small"

config.clj (defpallet :services { :default { :blobstore-provider "aws-s3" :provider "aws-ec2" :environment {:user {:username "storm" :private-key-path "/path/to/key.pem" :public-key-path "/path/to/key.pub"} :aws-user-id "xxxxxxxxxxxxxx"} :identity "xxxxxxx" :credential "xxxxxxx" :jclouds.regions "us-west-2" } })

Is there something obvious I'm missing? After getting this error, the process hangs and doesn't exit.

[elee]

You can specify the tag in master by specifying the SHA1 of the commit.

$ cd <incubator-storm checkout root>
$ git checkout master
$ git show-ref --tags
061ee2e7ca6ed89dbf0522ba3b2ccee6558c4105 refs/tags/0.9.0
449e4e4560f84e078a7d79f0b3159e22508ac228 refs/tags/0.9.0-rc1
32098d5b2694434ea43d430a4703fbe51bab268f refs/tags/0.9.0-rc2
66a397368a98834095b47db2b706420901c52ba3 refs/tags/0.9.0-rc3
1bcc169f5096e03a4ae117efc65c0f9bcfa2fa22 refs/tags/0.9.0.1
78d431828e88c35e55e2a7a9ad66d2ce6f8bcd07 refs/tags/v0.9.1-incubating
$

Find the SHA1 you want and supply that as an argument to --commit i.e. for release 0.9.0.1

$ lein deploy-storm --start --commit 1bcc169f5096e03a4ae117efc65c0f9bcfa2fa22

[ssolbak]

Ok that worked. All other v0.9 I was unable to get going. Thanks!

[dsampath]

I am seeing the same issue. Could I know what was it that worked? a. versions < 0.9 (or) b. version 0.9.01 ?

I am running the default topology on m1.Large and similar setup. Here is the last snippet from the logs :

2014-03-18 16:32:42,085 ERROR java.logging at pallet.core$raise_on_error$fn5204.invoke(core.clj:526) 2014-03-18 16:32:42,085 ERROR java.logging at pallet.core$middleware_handler$fn__5197.invoke(core.clj:495) 2014-03-18 16:32:42,085 ERROR java.logging at pallet.core$apply_phase_to_node.invoke(core.clj:656) 2014-03-18 16:32:42,085 ERROR java.logging at pallet.core$eval5367$fn5368$iter53695373$fn5374$fn5379.invoke(core.clj:721) 2014-03-18 16:32:42,085 ERROR java.logging at clojure.lang.AFn.call(AFn.java:18) 2014-03-18 16:32:42,085 ERROR java.logging at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 2014-03-18 16:32:42,085 ERROR java.logging at java.util.concurrent.FutureTask.run(FutureTask.java:138) 2014-03-18 16:32:42,085 ERROR java.logging at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) 2014-03-18 16:32:42,085 ERROR java.logging at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) 2014-03-18 16:32:42,086 ERROR java.logging at java.lang.Thread.run(Thread.java:695) 2014-03-18 16:32:42,114 ERROR pallet.execute Exit status : 1 2014-03-18 16:32:42,288 DEBUG clj-ssh.ssh Disconnecting from 54.80.117.158 port 22 2014-03-18 16:32:42,288 DEBUG [clj-ssh.ssh](Connect thread 54.80.117.158 session) Caught an exception, leaving main loop due to Socket closed 2014-03-18 16:32:42,343 ERROR pallet.core errors found [{:message "Error executing script :\n :cmd echo \"package-manager update ...\"\n{ aptitude update || true; }

[ssolbak]

Mine worked with 0.9.0.1

Make sure you have less then 20 ec2 instances per region. That's the default limit

Sent from my iPhone

On Mar 18, 2014, at 5:08 PM, Dhananjay Sampath notifications@github.com wrote:

I am seeing the same issue. Could I know what was it that worked? a. versions < 0.9 (or) b. version 0.9.01 ?

I am running the default topology on m1.Large

— Reply to this email directly or view it on GitHub.

[shouhengy-porch]

I have the same issue with deploying v0.9.1-incubating on AWS. .pallet.config.clj and clusters.yaml is very similar to ssolbak's. Mine worked with 0.9.0.1. Can somebody take a look? Thanks.

$ lein deploy-storm --start --name misato --branch master --commit 78d431828e88c35e55e2a7a9ad66d2ce6f8bcd07

[tbatchelli]

The way storm builds now has changed to maven3, and bin/build_release.sh does not exist anymore. I created issue #64 to cover this work.

[wl258]

Any update?