search

hortonworks / streamline

StreamLine - Streaming Analytics
Apache License 2.0
164 stars 96 forks source link

SAM "Druid Executer" in HDF 3.1 is not working under Storm 1.1.0 - but working with Storm 1.1.1 #1302

Closed emohdkamel closed 6 years ago

emohdkamel commented 6 years ago

After the building and the deployment of a simple app of SAM (HDF3.1) that has Druid Sink in Strom 1.1.0, there is an error continuously happening as following:

java.lang.NoSuchMethodError: org.apache.curator.utils.PathUtils.validatePath(Ljava/lang/String;)Ljava/lang/String; at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.<init>(InterProcessSemaphoreV2.java:119) at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.<init>(InterProcessSemaphoreV2.java:103) at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreMutex.<init>(InterProcessSemaphoreMutex.java:40) at com.metamx.tranquility.beam.ClusteredBeam.<init>(ClusteredBeam.scala:135) at com.metamx.tranquility.druid.DruidBeams$Builder.buildBeam(DruidBeams.scala:758) at com.hortonworks.streamline.streams.layout.storm.DruidBeamFactoryImpl.makeBeam(DruidBeamFactoryImpl.java:131) at org.apache.storm.druid.bolt.DruidBeamBolt.prepare(DruidBeamBolt.java:73) at org.apache.storm.daemon.executor$fn__10195$fn__10208.invoke(executor.clj:800) at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:482) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745)

However, when the same SAM app is deployed under Storm 1.1.1, the error is not appearing at all and the app works as it should be.

arunmahadevan commented 6 years ago

There are no changes in the curator versions between Storm v1.1.0 (https://github.com/apache/storm/blob/v1.1.0/pom.xml) and Storm v1.1.1 (https://github.com/apache/storm/blob/v1.1.1/pom.xml). Both has dependency on curator v2.12.0 which has the "String validatePath(String)" defined.

It appears that for some reason an older version of curator (<= 2.6) is getting included while you deploy with storm 1.1.0. You might want to check if thats the case.

emohdkamel commented 6 years ago

Thanks for your reply. Let me share some background; if it might help in understanding the issue. The error happens only when I'm using SAM 0.6.0 (in HDF3.1) to deploy under Storm v.1.1.0 (in HDP 2.6.x). However, the issue doesn't appear when using SAM 0.6.0 (in HDF3.1) to deploy under Storm v.1.1.1 (packed the same HDF 3.1 cluster).

HeartSaVioR commented 6 years ago

HDF on HDP is not supported or restricted until both have same Storm version. SAM is mainly targeted to work with HDF Storm. If SAM in HDF 3.1 works properly with Storm in HDF 3.1, no issue at all. I think we won't fix the issue.