search

yahoo / storm-yarn

Storm-yarn enables Storm clusters to be deployed into machines managed by Hadoop YARN.
Other
417 stars 161 forks source link

(Issue #56) Support Apache Hadoop 2.2.0 #57

Open anfeng opened 10 years ago

anfeng commented 10 years ago

Upgrade Storm-YARN to support Apache Hadoop 2.2.0 (instead of 2.1.0-beta)

goog commented 10 years ago

should -D ... the default hadoop.version is 2.1.0-beta

goog commented 10 years ago

the sample test seems work with hadoop2.2.0 when following http://kingshijie.blogspot.com/2013/10/step-by-step-deploying-storm-yarn-on.html . but the Storm UI monitoring tool doesn't . @anfeng could you give me some guide?

aperepel commented 10 years ago

Hi, the UI worked fine for my on HDP2.0 (and in multiple clusters). Ensure you point your browser to the right ip and port, though. Can you maybe post more details of how it's not working for you?​

Andrew

goog commented 10 years ago

i set the storm home at the master(.151) of apache hadoop2.2.0 , when visit http://192.168.0.152:7070/ , Internal Server Error:

org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
    at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
    at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:56)
    at backtype.storm.ui.core$main_page.invoke(core.clj:126)
    at backtype.storm.ui.core$fn__7848.invoke(core.clj:741)
    at compojure.core$make_route$fn__3840.invoke(core.clj:93)
    at compojure.core$if_route$fn__3828.invoke(core.clj:39)
    at compojure.core$if_method$fn__3821.invoke(core.clj:24)
    at compojure.core$routing$fn__3846.invoke(core.clj:106)
    at clojure.core$some.invoke(core.clj:2390)
    at compojure.core$routing.doInvoke(core.clj:106)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invoke(core.clj:603)
    at compojure.core$routes$fn__3850.invoke(core.clj:111)
    at clojure.lang.Var.invoke(Var.java:415)
    at ring.middleware.reload$wrap_reload$fn__7144.invoke(reload.clj:14)
    at backtype.storm.ui.core$catch_errors$fn__7886.invoke(core.clj:798)
    at ring.adapter.jetty$proxy_handler$fn__4189.invoke(jetty.clj:16)
    at ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
    ... 27 more

is it needed to install the zookeeper at hadoop master and slave nodes? BTW: what's meaning of ''You need to install a version of storm on the hadoop gateway.'' , i have deployed storm.zip on the master node. is there something i am missing ?

goog commented 10 years ago

with the zookeeper installed , solved it .

aperepel commented 10 years ago

Yes, this is typically an error when ui cannot communicate with zookeeper quorum to locate other storm components. Should be in a FAQ ;)

Andrew On Dec 18, 2013 1:20 AM, "AsFlowers" notifications@github.com wrote:

with the zookeeper installed , solved it .

— Reply to this email directly or view it on GitHubhttps://github.com/yahoo/storm-yarn/pull/57#issuecomment-30819707 .