Deploy fails, 'connection refused'

[yaronr]

Hi

I'm using boot2docker. fig up. Storm UI working @ http://192.168.59.103:49080/ I tried: storm jar target/my.jar my.poackage.Topology my-topology -c nimbus.host=192.168.59.103 --> connection refused storm jar target/my.jar my.poackage.Topology my-topology -c nimbus.host=192.168.59.103:49627 --> UnknownHostException storm jar target/my.jar my.poackage.Topology my-topology 192.168.59.103 --> Connection refused

I'm obviously doing something wrong, but what? :-)

PS I looked at: https://github.com/wurstmeister/storm-kafka-0.8-plus-test I don't understand what you mean by - and how it relates to the default storm-docker (without kafka). Is it the IP:port of the zookeeper?

I must say I checked the storm documentation as well: http://storm.incubator.apache.org/documentation/Command-line-client.html which just states : Syntax: storm jar topology-jar-path class ... without explaining what '...' includes

Thank you very much for your patience.

[wurstmeister]

you should specify the port separately: -c nimbus.port=49627 (or try https://github.com/wurstmeister/storm-docker/issues/1#issuecomment-51302374)

[yaronr]

storm jar target/my.jar com.my.package.Topology foo -c nimbus.host=192.168.59.103 -c nimbus.port=49627 --> 'Connection refused'.

This looks like the correct approach, as opposed to #1 comment (and much simpler) - only it doesn't work for me.

Can you think of something else I can do?

[wurstmeister]

Hi,

sorry the parameter is actually called

nimbus.thrift.port

unfortunately is seems it can't be overwritten with -c

field nimbus.thrift.port '49627' must be a 'java.lang.Number'

you can create

~/.storm/storm.yaml

and set the parameter in there:

nimbus.host: "192.168.59.103"
nimbus.thrift.port: 49627

alternatively you can set it in the topology config:

config.put(Config.NIMBUS_HOST,  "192.168.59.103");
config.put(Config.NIMBUS_THRIFT_PORT, 49627);

[yaronr]

Haleluja It finally works. Thank you very much! @wurstmeister, I would like to help other people that may have the same questions, save all the time that you and I both spent trying to solve this. Would you like me to start an FAQ section in the readme or something of the sort? Also, what do you think about asking for a feature to enable command line override of the Thrift port in storm in the storm project, would you like to do it or would you prefer that I open the request?

[wurstmeister]

Hi,

looks like I had an old version of storm on my $PATH, the error regarding the port configuration has been fixed in storm 0.9.2 (https://issues.apache.org/jira/browse/STORM-173)

./apache-storm-0.9.2-incubating/bin/storm list -c nimbus.host=localhost -c nimbus.thrift.port=1235

1273 [main] INFO  backtype.storm.thrift - Connecting to Nimbus at localhost:1235

I'd be happy to accept a PR for an FAQ section!

Thanks

[yaronr]

Will do!

But you have to tell me how you did it. I just ran fig up, and suddenly it works. I didn't update the git, I saw that the docker files are the same, .yaml file is the same nothing was changed in the repository

So how come it suddenly works?! :)

(Y)

On Aug 9, 2014, at 6:46 PM, wurstmeister notifications@github.com wrote:

Hi,

looks like I had an old version of storm on my $PATH, the error regarding the port configuration has been fixed in storm 0.9.2 (https://issues.apache.org/jira/browse/STORM-173)

./apache-storm-0.9.2-incubating/bin/storm list -c nimbus.host=localhost -c nimbus.thrift.port=1235 1273 [main] INFO backtype.storm.thrift - Connecting to Nimbus at localhost:1235 I'd be happy to accept a PR for an FAQ section!

Thanks

— Reply to this email directly or view it on GitHub.

[yaronr]

Committed 58c84ee

Let me know how you did this magic of fixing the problem without any file changing :) (and let me know if my commit is OK, this is the first time I commit on GIT. I usually use SVN)

(Y)

On Aug 9, 2014, at 6:46 PM, wurstmeister notifications@github.com wrote:

Hi,

looks like I had an old version of storm on my $PATH, the error regarding the port configuration has been fixed in storm 0.9.2 (https://issues.apache.org/jira/browse/STORM-173)

./apache-storm-0.9.2-incubating/bin/storm list -c nimbus.host=localhost -c nimbus.thrift.port=1235 1273 [main] INFO backtype.storm.thrift - Connecting to Nimbus at localhost:1235 I'd be happy to accept a PR for an FAQ section!

Thanks

— Reply to this email directly or view it on GitHub.

[yaronr]

Ok,

I just found that I still had my ~/.storm/storm.yaml. After removing it, the magic dissipated, and I was no longer able to deploy - 'connection refused'. My storm version is 0.9.2-incubating

So it's not your $PATH I suspect... Or am I missing something?

(Y)

On Aug 10, 2014, at 8:59 AM, Yaron Rosenbaum yaron.rosenbaum@gmail.com wrote:

Will do!

But you have to tell me how you did it. I just ran fig up, and suddenly it works. I didn't update the git, I saw that the docker files are the same, .yaml file is the same nothing was changed in the repository

So how come it suddenly works?! :)

(Y)

On Aug 9, 2014, at 6:46 PM, wurstmeister notifications@github.com wrote:

Hi,

looks like I had an old version of storm on my $PATH, the error regarding the port configuration has been fixed in storm 0.9.2 (https://issues.apache.org/jira/browse/STORM-173)

./apache-storm-0.9.2-incubating/bin/storm list -c nimbus.host=localhost -c nimbus.thrift.port=1235 1273 [main] INFO backtype.storm.thrift - Connecting to Nimbus at localhost:1235 I'd be happy to accept a PR for an FAQ section!

Thanks

— Reply to this email directly or view it on GitHub.

[wurstmeister]

can you please post the exact command you are trying to execute together with the complete output?

[wurstmeister]

can you please try:

storm list -c nimbus.host=192.168.59.103 -c nimbus.thrift.port=49627

and post the full output.

also can you please post the content of your ~/.storm/storm.yaml and confirm that if you use the yaml instead of the command line arguments it works?

i.e. please put ~/.storm/storm.yaml in place and run

storm list

and post the output

[yaronr]

Same. connection refused: YaronR-MBP:storm yaron$ storm list -c nimbus.host=192.168.59.103 -c nimbus.thrift.port=49627 Running: java -client -Dstorm.options=nimbus.host%3D192.168.59.103,nimbus.thrift.port%3D49627 -Dstorm.home=/usr/local/Cellar/storm/storm/libexec -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/Cellar/storm/storm/libexec/lib/asm-4.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/carbonite-1.4.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/chill-java-0.3.5.jar:/usr/local/Cellar/storm/storm/libexec/lib/clj-stacktrace-0.2.4.jar:/usr/local/Cellar/storm/storm/libexec/lib/clj-time-0.4.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/clojure-1.5.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/clout-1.0.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/commons-codec-1.6.jar:/usr/local/Cellar/storm/storm/libexec/lib/commons-exec-1.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/commons-fileupload-1.2.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/commons-io-2.4.jar:/usr/local/Cellar/storm/storm/libexec/lib/commons-lang-2.5.jar:/usr/local/Cellar/storm/storm/libexec/lib/commons-logging-1.1.3.jar:/usr/local/Cellar/storm/storm/libexec/lib/compojure-1.1.3.jar:/usr/local/Cellar/storm/storm/libexec/lib/core.incubator-0.1.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/curator-client-2.4.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/curator-framework-2.4.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/disruptor-2.10.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/guava-13.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/hiccup-0.3.6.jar:/usr/local/Cellar/storm/storm/libexec/lib/httpclient-4.3.3.jar:/usr/local/Cellar/storm/storm/libexec/lib/httpcore-4.3.2.jar:/usr/local/Cellar/storm/storm/libexec/lib/jetty-6.1.26.jar:/usr/local/Cellar/storm/storm/libexec/lib/jetty-util-6.1.26.jar:/usr/local/Cellar/storm/storm/libexec/lib/jgrapht-core-0.9.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/jline-2.11.jar:/usr/local/Cellar/storm/storm/libexec/lib/joda-time-2.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/json-simple-1.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/kryo-2.21.jar:/usr/local/Cellar/storm/storm/libexec/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/Cellar/storm/storm/libexec/lib/logback-classic-1.0.6.jar:/usr/local/Cellar/storm/storm/libexec/lib/logback-core-1.0.6.jar:/usr/local/Cellar/storm/storm/libexec/lib/math.numeric-tower-0.0.1.jar:/usr/local/Cellar/storm/storm/libexec/lib/minlog-1.2.jar:/usr/local/Cellar/storm/storm/libexec/lib/netty-3.2.2.Final.jar:/usr/local/Cellar/storm/storm/libexec/lib/netty-3.6.3.Final.jar:/usr/local/Cellar/storm/storm/libexec/lib/objenesis-1.2.jar:/usr/local/Cellar/storm/storm/libexec/lib/reflectasm-1.07-shaded.jar:/usr/local/Cellar/storm/storm/libexec/lib/ring-core-1.1.5.jar:/usr/local/Cellar/storm/storm/libexec/lib/ring-devel-0.3.11.jar:/usr/local/Cellar/storm/storm/libexec/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/Cellar/storm/storm/libexec/lib/ring-servlet-0.3.11.jar:/usr/local/Cellar/storm/storm/libexec/lib/servlet-api-2.5-20081211.jar:/usr/local/Cellar/storm/storm/libexec/lib/servlet-api-2.5.jar:/usr/local/Cellar/storm/storm/libexec/lib/slf4j-api-1.6.5.jar:/usr/local/Cellar/storm/storm/libexec/lib/snakeyaml-1.11.jar:/usr/local/Cellar/storm/storm/libexec/lib/storm-core-0.9.2-incubating.jar:/usr/local/Cellar/storm/storm/libexec/lib/tools.cli-0.2.4.jar:/usr/local/Cellar/storm/storm/libexec/lib/tools.logging-0.2.3.jar:/usr/local/Cellar/storm/storm/libexec/lib/tools.macro-0.1.0.jar:/usr/local/Cellar/storm/storm/libexec/lib/zookeeper-3.4.5.jar:/usr/local/Cellar/storm/storm/libexec/conf:/usr/local/Cellar/storm/storm/libexec/bin backtype.storm.command.list 2057 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 192.168.59.103:49627 Exception in thread "main" org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183) at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81) at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75) at backtype.storm.command.list$_main.invoke(list.clj:22) at clojure.lang.AFn.applyToHelper(AFn.java:159) at clojure.lang.AFn.applyTo(AFn.java:151) at backtype.storm.command.list.main(Unknown Source) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift7.transport.TSocket.open(TSocket.java:178) ... 6 more

YaronR-MBP:storm yaron$ mv ~/.storm/storm.yaml_tmp ~/.storm/storm.yaml YaronR-MBP:storm yaron$ cat ~/.storm/storm.yaml nimbus.host: "192.168.59.103" nimbus.thrift.port: 49627

and

YaronR-MBP:storm yaron$ storm list -c nimbus.host=192.168.59.103 -c nimbus.thrift.port=49627 ... 1343 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 192.168.59.103:49627 No topologies running.

works

(Y)

On Aug 10, 2014, at 12:14 PM, wurstmeister notifications@github.com wrote:

can you please try:

storm list -c nimbus.host=192.168.59.103 -c nimbus.thrift.port=49627 and post the full output.

also can you please post the content of your ~/.storm/storm.yaml and confirm that if you use the yaml instead of the command line arguments it works?

i.e. please put ~/.storm/storm.yaml in place and run

storm list and post the output

— Reply to this email directly or view it on GitHub.

[wurstmeister]

based on your output you are trying to connect to the same host:port in both cases

2057 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 192.168.59.103:49627

1343 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 192.168.59.103:49627

so passing the -c nimbus.thrift.port parameter seems to work ok (you don't need that parameter when you have the yaml file in place)

I can't reproduce your problem, both approaches work ok. It would be interesting to see what happens if you use the default port in the fig.yml

- "6627:6627"

[yaronr]

Strange, but I can make it work, and that's what matters :)

BTW. Let me know if my commit is OK (this is my first time ever to commit in GIT). BTW 2. I'm working on a Mesos+storm fig setup, let me know if you're intrigued.

Thank you very much Yaron

(Y)

On Aug 10, 2014, at 1:30 PM, wurstmeister notifications@github.com wrote:

based on your output you are trying to connect to the same host:port in both cases

2057 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 192.168.59.103:49627 1343 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 192.168.59.103:49627 so passing the -c nimbus.thrift.port parameter seems to work ok (you don't need that parameter when you have the yaml file in place)

I can't reproduce your problem, both approaches work ok. It would be interesting to see what happens if you use the default port in the fig.yml

• "6627:6627" — Reply to this email directly or view it on GitHub.

[yaronr]

Hi

FYI, inspired by your storm-docker project, I created a Mesos (+Storm) setup on github: https://github.com/yaronr/docker-mesos

I liked the simplicity and elegance of fig + docker, and the dev-ops implications (especially resource consumption) - so my shared project uses the same approach (albeit less elegantly :) )

It would be great if you could take a look and maybe contribute.

Cheers, (Y)

[wurstmeister]

Thanks. I'll have a look.

Regarding the commit for the readme, could you please send a pull request (https://help.github.com/articles/using-pull-requests)

Thanks

[yaronr]

Done. I think :) Git starts to make sense, slowly.

I am creating a group of very strong professionals that will contribute to this project. Your help on docker-mesos will be greatly appreciated, and you will enjoy collaborating with some of these people.

Let me know what you think.

(Y)

On Aug 13, 2014, at 12:44 AM, wurstmeister notifications@github.com wrote:

Thanks. I'll have a look.

Regarding the commit for the readme, could you please send a pull request (https://help.github.com/articles/using-pull-requests)

Thanks

— Reply to this email directly or view it on GitHub.

[Ali-Alhaboby]

Problem with Apache Storm : connection refused I'm facing this problem when I have installed Apache Storm on two machines on AWS EC2 Amazon storm list Running: java -client -Dstorm.options=nimbus.host%3D172.31.32.97,nimbus.thrift.port%3D6627 -Dstorm.home=/usr/local/src/apache-storm-0.9.4 -Dstorm.log.dir=/usr/local/src/apache-storm-0.9.4/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/src/apache-storm-0.9.4/lib/commons-codec-1.6.jar:/usr/local/src/apache-storm-0.9.4/lib/json-simple-1.1.jar:/usr/local/src/apache-storm-0.9.4/lib/storm-core-0.9.4.jar:/usr/local/src/apache-storm-0.9.4/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/src/apache-storm-0.9.4/lib/hiccup-0.3.6.jar:/usr/local/src/apache-storm-0.9.4/lib/carbonite-1.4.0.jar:/usr/local/src/apache-storm-0.9.4/lib/tools.macro-0.1.0.jar:/usr/local/src/apache-storm-0.9.4/lib/ring-servlet-0.3.11.jar:/usr/local/src/apache-storm-0.9.4/lib/commons-logging-1.1.3.jar:/usr/local/src/apache-storm-0.9.4/lib/servlet-api-2.5.jar:/usr/local/src/apache-storm-0.9.4/lib/joda-time-2.0.jar:/usr/local/src/apache-storm-0.9.4/lib/commons-lang-2.5.jar:/usr/local/src/apache-storm-0.9.4/lib/asm-4.0.jar:/usr/local/src/apache-storm-0.9.4/lib/disruptor-2.10.1.jar:/usr/local/src/apache-storm-0.9.4/lib/jetty-util-6.1.26.jar:/usr/local/src/apache-storm-0.9.4/lib/clj-stacktrace-0.2.2.jar:/usr/local/src/apache-storm-0.9.4/lib/tools.cli-0.2.4.jar:/usr/local/src/apache-storm-0.9.4/lib/math.numeric-tower-0.0.1.jar:/usr/local/src/apache-storm-0.9.4/lib/ring-devel-0.3.11.jar:/usr/local/src/apache-storm-0.9.4/lib/reflectasm-1.07-shaded.jar:/usr/local/src/apache-storm-0.9.4/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/src/apache-storm-0.9.4/lib/ring-core-1.1.5.jar:/usr/local/src/apache-storm-0.9.4/lib/commons-exec-1.1.jar:/usr/local/src/apache-storm-0.9.4/lib/kryo-2.21.jar:/usr/local/src/apache-storm-0.9.4/lib/logback-core-1.0.13.jar:/usr/local/src/apache-storm-0.9.4/lib/logback-classic-1.0.13.jar:/usr/local/src/apache-storm-0.9.4/lib/commons-fileupload-1.2.1.jar:/usr/local/src/apache-storm-0.9.4/lib/clojure-1.5.1.jar:/usr/local/src/apache-storm-0.9.4/lib/jline-2.11.jar:/usr/local/src/apache-storm-0.9.4/lib/chill-java-0.3.5.jar:/usr/local/src/apache-storm-0.9.4/lib/clout-1.0.1.jar:/usr/local/src/apache-storm-0.9.4/lib/clj-time-0.4.1.jar:/usr/local/src/apache-storm-0.9.4/lib/objenesis-1.2.jar:/usr/local/src/apache-storm-0.9.4/lib/compojure-1.1.3.jar:/usr/local/src/apache-storm-0.9.4/lib/snakeyaml-1.11.jar:/usr/local/src/apache-storm-0.9.4/lib/minlog-1.2.jar:/usr/local/src/apache-storm-0.9.4/lib/commons-io-2.4.jar:/usr/local/src/apache-storm-0.9.4/lib/jgrapht-core-0.9.0.jar:/usr/local/src/apache-storm-0.9.4/lib/jetty-6.1.26.jar:/usr/local/src/apache-storm-0.9.4/lib/core.incubator-0.1.0.jar:/usr/local/src/apache-storm-0.9.4/lib/slf4j-api-1.7.5.jar:/usr/local/src/apache-storm-0.9.4/lib/tools.logging-0.2.3.jar:/usr/local/src/apache-storm-0.9.4/conf:/usr/local/src/apache-storm-0.9.4/bin backtype.storm.command.list 53149 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 172.31.32.97:6627 Exception in thread "main" org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183) at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81) at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75) at backtype.storm.command.list$_main.invoke(list.clj:22) at clojure.lang.AFn.applyToHelper(AFn.java:159) at clojure.lang.AFn.applyTo(AFn.java:151) at backtype.storm.command.list.main(Unknown Source) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.thrift7.transport.TSocket.open(TSocket.java:178) ... 6 more