storm-deploy exception

[BenMacKenzie]

I've tried running storm-deploy a few times and it is consistently failing both from my laptop (osx) and from an ec2 rhel 6.4 instance. The exception occurs directly after storm-deploy appears to be starting ganglia on all the nodes. The last few messages before the exception are as follows:

DEBUG core - p-f-s server environment null DEBUG core - p-f-s server environment null INFO core - parallel-apply-phase :pallet.phase/post-exec for :supervisor-stormClusterX INFO core - parallel-apply-phase-to-target :node :pallet.phase/post-exec for :supervisor-stormClusterX with 2 nodes DEBUG core - apply-phase-to-node: phase :pallet.phase/post-exec group :supervisor-stormClusterX target 54.226.164.194 DEBUG core - apply-phase-to-node: phase :pallet.phase/post-exec group :supervisor-stormClusterX target 54.224.119.37 INFO execute - execute-with-ssh on supervisor-stormClusterX "54.226.164.194" INFO execute - execute-with-ssh on supervisor-stormClusterX "54.224.119.37" INFO execute - Admin user storm /Users/benmackenzie/.ssh-aws/BenMac.pem /Users/benmackenzie/.ssh-aws/BenMac.pem.pub INFO execute - Admin user storm /Users/benmackenzie/.ssh-aws/BenMac.pem /Users/benmackenzie/.ssh-aws/BenMac.pem.pub DEBUG provision - Finished post-configure and exec phases INFO provision - Attaching to Available Cluster... DEBUG compute - >> listing node details matching(ALWAYS_TRUE) DEBUG compute - << list(10) DEBUG compute - >> listing node details matching(ALWAYS_TRUE) DEBUG compute - << list(10) DEBUG compute - >> listing node details matching(ALWAYS_TRUE) DEBUG compute - << list(10) DEBUG compute - >> listing node details matching(ALWAYS_TRUE) DEBUG compute - << list(10) ERROR logging - Exception in thread "main" ERROR logging - java.lang.RuntimeException: request: POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1; cause: java.lang.NullPointerException (NO_SOURCE_FILE:1)

**config.clj is as follows:

(defpallet :services { :default { :blobstore-provider "aws-s3" :provider "aws-ec2" :environment {:user {:username "storm" ; this must be "storm" :private-key-path "~/.ssh-aws/BenMac.pem" :public-key-path "~/.ssh-aws/BenMac.pem.pub"} :aws-user-id "xxxxxx"} :identity "xxxxxxx" :credential "xxxxxxx" :jclouds.regions "us-east-1" } })

any ideas?

[BenMacKenzie]

Just a quick follow up:

notwithstanding the fact the storm-deploy threw an exception and did not complete, I am able to submit and execute a topology.

Also, I had to manually modify the aws security group nimbus is assigned to in order to access the ui.

[jsquirrelz]

I'm having the same issue. As soon as the attaching phase begins, it throws a RuntimeException:

ERROR logging - Exception in thread "main" ERROR logging - java.lang.RuntimeException: request: POST https://ec2.us-west-1.amazonaws.com/ HTTP/1.1; cause: java.lang.NullPointerException (NO_SOURCE_FILE:1)

Unfortunately I made the edit to the Nimbus security group (added port 8080) and tried to visit the UI, but it threw an internal server error. Tried submitting a topology as well, turns out Nimbus never started (or crashed on startup?).

[rjtg]

Same issue for me

[gworley3]

I wrote a reply to this on the mailing list: something changed with Amazon's security group api responses because it is breaking the version of jclouds used in storm-deploy. They made the change a couple weeks ago. Not sure what has changed that is incompatible with the version of jclouds used by storm-deploy, but a workaround that we're using is to comment out everything that sets security groups and manage them manually via the ec2 web ui after we bring up the cluster.

[kerinin]

Same here