search

OpenTSDB / opentsdb

A scalable, distributed Time Series Database.
http://opentsdb.net
GNU Lesser General Public License v2.1
5k stars 1.25k forks source link

Cannot run OpenTSDB 3 in Docker because of dependency error (Netty, NoSuchMethodError) #1918

Closed gregory112 closed 3 years ago

gregory112 commented 4 years ago

I have built OpenTSDB 3 in Docker with mvn package -Pdocker as instructed on the docs and was able to run it. But during startup I got this error:

12:54:27.599 INFO  [PluginsConfig.registerPlugin] - Registered plugin id=LRUUniqueId, type=net.opentsdb.uid.UniqueIdFactory, plugin=net.opentsdb.uid.LRUUniqueIdFactory, isDefault=false
12:54:27.689 INFO  [PluginsConfig.registerPlugin] - Registered plugin id=JsonV2ExpQuerySerdes, type=net.opentsdb.query.serdes.SerdesFactory, plugin=net.opentsdb.query.execution.serdes.JsonV2ExpQuerySerdesFactory, isDefault=false
12:54:27.689 INFO  [PluginsConfig.registerPlugin] - Registered plugin id=JsonV3QuerySerdes, type=net.opentsdb.query.serdes.SerdesFactory, plugin=net.opentsdb.query.execution.serdes.JsonV3QuerySerdesFactory, isDefault=false
12:54:27.689 INFO  [PluginsConfig.registerPlugin] - Registered plugin id=JsonV2QuerySerdes, type=net.opentsdb.query.serdes.SerdesFactory, plugin=net.opentsdb.query.execution.serdes.JsonV2QuerySerdesFactory, isDefault=false
12:54:27.778 INFO  [PluginsConfig.registerPlugin] - Registered plugin id=TSDBServletSink, type=net.opentsdb.query.QuerySinkFactory, plugin=net.opentsdb.servlet.sinks.ServletSinkFactory, isDefault=false
12:54:27.779 INFO  [PluginsConfig.call] - Loaded plugin null version: 3.0.0
12:54:27.779 INFO  [BaseQueryIntperolatorFactory.register] - Stored interpolator builder for type net.opentsdb.data.types.numeric.NumericType in null Class: class net.opentsdb.query.interpolation.types.numeric.NumericInterpolator
12:54:27.779 INFO  [BaseQueryIntperolatorFactory.register] - Stored interpolator builder for type net.opentsdb.data.types.numeric.NumericSummaryType in null Class: class net.opentsdb.query.interpolation.types.numeric.NumericSummaryInterpolator
12:54:27.780 INFO  [PluginsConfig.registerPlugin] - Registered plugin id=null, type=net.opentsdb.query.interpolation.QueryInterpolatorFactory, plugin=net.opentsdb.query.interpolation.DefaultInterpolatorFactory, isDefault=true
12:54:27.780 INFO  [PluginsConfig.call] - Loaded plugin null version: 3.0.0
12:54:27.780 INFO  [PluginsConfig.registerPlugin] - Registered plugin id=null, type=net.opentsdb.storage.schemas.tsdb1x.Tsdb1xDataStoreFactory, plugin=net.opentsdb.storage.Tsdb1xHBaseFactory, isDefault=true
12:54:27.781 INFO  [PluginsConfig.call] - Loaded plugin null version: 3.0.0
12:54:27.806 ERROR [TSDMain.main] - Failed to initialize TSDB registry
java.lang.NoSuchMethodError: org.jboss.netty.util.HashedWheelTimer.<init>(Ljava/util/concurrent/ThreadFactory;Lorg/jboss/netty/util/ThreadNameDeterminer;JLjava/util/concurrent/TimeUnit;I)V
    at org.hbase.async.HBaseClient.newTimer(HBaseClient.java:737)
    at org.hbase.async.HBaseClient.defaultChannelFactory(HBaseClient.java:765)
    at org.hbase.async.HBaseClient.<init>(HBaseClient.java:628)
    at net.opentsdb.storage.Tsdb1xHBaseDataStore.<init>(Tsdb1xHBaseDataStore.java:301)
    at net.opentsdb.storage.Tsdb1xHBaseFactory.newInstance(Tsdb1xHBaseFactory.java:97)
    at net.opentsdb.storage.schemas.tsdb1x.Schema.<init>(Schema.java:148)
    at net.opentsdb.storage.schemas.tsdb1x.SchemaFactory.initialize(SchemaFactory.java:105)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:467)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:473)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:473)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
    at com.stumbleupon.async.Deferred.addCallbacks(Deferred.java:685)
    at com.stumbleupon.async.Deferred.addCallback(Deferred.java:721)
    at net.opentsdb.core.PluginsConfig$1SuccessCB.call(PluginsConfig.java:522)
    at net.opentsdb.core.PluginsConfig.initialize(PluginsConfig.java:561)
    at net.opentsdb.core.DefaultRegistry.loadPlugins(DefaultRegistry.java:448)
    at net.opentsdb.core.DefaultRegistry.initialize(DefaultRegistry.java:167)
    at net.opentsdb.core.DefaultTSDB.initializeRegistry(DefaultTSDB.java:393)
    at net.opentsdb.tsd.TSDMain.main(TSDMain.java:202)

This seems to be related to Netty. Checking the content of the image, Netty 3.2.2, 3.9.4, and 4.1.9 jars are all loaded to /usr/share/opentsdb/lib with CLASSPATH args set correctly in bin/tsdb script. I am not an expert in Java but this stacktrace said that it is calling a constructor named HashedWheelTimer with three parameters: ThreadFactory, ThreadNameDeterminer, TimeUnit yes? The stacktrace said it is calling org.jboss.netty which is Netty version 3 (version 4 is io.netty). However, there is no HashedWheelTimer constructor with that signature in Netty version 3 or 4.

Anyway I could be wrong. But this seems to be an incorrect jar issue and because it is run inside Docker, it does not share CLASSPATH with other installations like many did. This error is common in OpenTSDB 2 it seems due to people mixing CLASSPATH from HBase installation and used incorrect Netty version from HBase instead of the one shipped with OpenTSDB. But this one lives inside Docker container without any other.

I am using OpenTSDB 3.0.90-SNAPSHOT, from branch 3.0 commit revision b4d67b2 (latest)

manolama commented 3 years ago

Sorry for the trouble, I had probably goofed up the container imports at the time. Now it's running properly with 3.9.4 and 4.1.9. We still have ES in there and I believe that is pulling from the older netty. I'll close this for now, re-open if you still see it off the 3.x main branch please.