apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.39k stars 3.68k forks source link

druid extensions-contrib deep storage cassandra loading error #7387

Closed Freya0214 closed 5 years ago

Freya0214 commented 5 years ago

was trying out cassandra deep storage with druid 0.13.0, cassandra database version is 3.11.3. The "common.runtime.properties file" has been added : druid.extensions.loadList=["druid-cassandra-storage", "druid-histogram", "druid-datasketches", "druid-kafka-indexing-service", "druid-parser-route", "imply-utility-belt"] Deep storage druid.storage.type=c* druid.storage.host=ssd78:9160 druid.storage.keyspace=druid

indexing-logs out the following exception:

ERROR [main] org.apache.druid.cli.CliPeon - Error when starting up. Failing. com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) Unknown provider[c*] of Key[type=org.apache.druid.segment.loading.DataSegmentPusher, annotation=[none]], known options[[local]] at org.apache.druid.guice.PolyBind.createChoice(PolyBind.java:70) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.guice.LocalDataStorageDruidModule) while locating org.apache.druid.segment.loading.DataSegmentPusher for the 4th parameter of org.apache.druid.indexing.common.TaskToolboxFactory.(TaskToolboxFactory.java:113) at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:201) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1) while locating org.apache.druid.indexing.common.TaskToolboxFactory for the 1st parameter of org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner.(SingleTaskBackgroundRunner.java:95) at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:240) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1) while locating org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner while locating org.apache.druid.indexing.overlord.TaskRunner for the 4th parameter of org.apache.druid.indexing.worker.executor.ExecutorLifecycle.(ExecutorLifecycle.java:79) at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:224) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1) while locating org.apache.druid.indexing.worker.executor.ExecutorLifecycle

1 error at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1028) ~[guice-4.1.0.jar:?] at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050) ~[guice-4.1.0.jar:?] at org.apache.druid.guice.LifecycleModule$2.start(LifecycleModule.java:132) ~[druid-api-0.13.0-incubating-iap9.jar:0.13.0-incubating-iap9] at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:109) [druid-services-0.13.0-incubating-iap9.jar:0.13.0-incubating-iap9] at org.apache.druid.cli.CliPeon.run(CliPeon.java:350) [druid-services-0.13.0-incubating-iap9.jar:0.13.0-incubating-iap9] at org.apache.druid.cli.Main.main(Main.java:118) [druid-services-0.13.0-incubating-iap9.jar:0.13.0-incubating-iap9]

hueiyuanSu commented 5 years ago

you can try to set this config in "common.runtime.properties file": druid.extensions.pullList = ["io.druid.extensions.contrib:druid-cassandra-storage:0.12.1"]

it should be work!

Freya0214 commented 5 years ago

you can try to set this config in "common.runtime.properties file": druid.extensions.pullList = ["io.druid.extensions.contrib:druid-cassandra-storage:0.12.1"] it should be work!

THX,I tried it,it's still not work, common.runtime.properties file" has been set : druid.extensions.directory=dist/druid/extensions druid.extensions.hadoopDependenciesDir=dist/druid/hadoop-dependencies druid.extensions.loadList=["druid-cassandra-storage", "druid-histogram", "druid-datasketches", "druid-kafka-indexing-service", "druid-parser-route", "imply-utility-belt"] druid.extensions.pullList=["io.druid.extensions.contrib:druid-cassandra-storage:0.12.0"]

indexing-logs out the following exception still unchanged

2019-04-01T17:07:37,674 ERROR [main] org.apache.druid.cli.CliPeon - Error when starting up. Failing. com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) Unknown provider[c*] of Key[type=org.apache.druid.segment.loading.DataSegmentPusher, annotation=[none]], known options[[local]] at org.apache.druid.guice.PolyBind.createChoice(PolyBind.java:70) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.guice.LocalDataStorageDruidModule) while locating org.apache.druid.segment.loading.DataSegmentPusher for the 4th parameter of org.apache.druid.indexing.common.TaskToolboxFactory.(TaskToolboxFactory.java:113) at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:201) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1) while locating org.apache.druid.indexing.common.TaskToolboxFactory for the 1st parameter of org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner.(SingleTaskBackgroundRunner.java:95) at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:240) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1) while locating org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner while locating org.apache.druid.indexing.overlord.TaskRunner for the 4th parameter of org.apache.druid.indexing.worker.executor.ExecutorLifecycle.(ExecutorLifecycle.java:79) at org.apache.druid.cli.CliPeon$1.configure(CliPeon.java:224) (via modules: com.google.inject.util.Modules$OverrideModule -> com.google.inject.util.Modules$OverrideModule -> org.apache.druid.cli.CliPeon$1) while locating org.apache.druid.indexing.worker.executor.ExecutorLifecycle

1 error at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1028) ~[guice-4.1.0.jar:?] at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1050) ~[guice-4.1.0.jar:?] at org.apache.druid.guice.LifecycleModule$2.start(LifecycleModule.java:132) ~[druid-api-0.13.0-incubating-iap11.jar:0.13.0-incubating-iap11] at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:109) [druid-services-0.13.0-incubating-iap11.jar:0.13.0-incubating-iap11] at org.apache.druid.cli.CliPeon.run(CliPeon.java:350) [druid-services-0.13.0-incubating-iap11.jar:0.13.0-incubating-iap11] at org.apache.druid.cli.Main.main(Main.java:118) [druid-services-0.13.0-incubating-iap11.jar:0.13.0-incubating-iap11]

wc188996 commented 5 years ago

I've met the question as you too!Is it a bug?

egor-ryashin commented 5 years ago

It didn't load the extension, as it says it doesn't know anything except local

known options[[local]]

Check the extension is present, ie find ./extensions

Freya0214 commented 5 years ago

It didn't load the extension, as it says it doesn't know anything except local

known options[[local]]

Check the extension is present, ie find ./extensions THX ! In the extension directory : image The “druid-cassandra-storage” Existing in the directory,"common.runtime.properties" file has been set path druid.extensions.directory=dist/druid/extensions I think Should not be unable to find this directory .

egor-ryashin commented 5 years ago

I see it's an Imply distributive, I think it's not an issue of Apache Druid project but of Imply project then.

wc188996 commented 5 years ago

I see it's an Imply distributive, I think it's not an issue of Apache Druid project but of Imply project then.

Hi,the problem is applies to Apache Druid project,too. The extension directory of Druid:

屏幕快照 2019-04-08 09 08 20

The configuration of "druid-cassandra-storage"

屏幕快照 2019-04-08 09 06 35

and the error is occurred!

屏幕快照 2019-04-08 09 35 56
wc188996 commented 5 years ago

you can try to set this config in "common.runtime.properties file": druid.extensions.pullList = ["io.druid.extensions.contrib:druid-cassandra-storage:0.12.1"]

it should be work!

Hi,I wanna know the version of Apache Druid which is running with the "druid-cassandra-storage" ? 0.12.1? Thanks! Maybe "druid-cassandra-storage" 's version is not match Apache Druid 's version?

wc188996 commented 5 years ago

This is a version matching issue. The "druid-cassandra-storage:0.12.x" does not match the version of the Apache Druid v0.13.0. Thank you very much, egor-ryashin This error no longer appears in my Druid index log,but there is a new error #7426 like yours .

Freya0214 commented 5 years ago

This is a version matching issue. The "druid-cassandra-storage:0.12.x" does not match the version of the Apache Druid v0.13.0. Thank you very much, egor-ryashin This error no longer appears in my Druid index log,but there is a new error #7426 like yours .

Yeah, it's that, fixing makes work for me 👍