Closed NikitaElmanov closed 2 years ago
NikitaElmanov
commented
2 years ago Does not help adding druid-moving-average-query.jar to /root/apache-druid-0.22.1/extensions and /root/apache-druid-0.22.1/libs by using workaround (use scp command). After restart druid "queryType": "movingAverage" is not available (issue: unsupported query type).
rohangarg
commented
2 years ago @NikitaElmanov : For the manual installation, did you do the following :
extensions/druid-moving-average-query folder in the deploymentdruid-moving-average-query-0.22.1.jar to the folder createdFor pulling the moving average jar from your custom repositories instead of repo1.maven I think you can try remote repository options at https://druid.apache.org/docs/latest/operations/pull-deps.html in the pull-deps command line - they allow you to specify a custom repository and use only that repository for searching the artifact.
Further, I think for setup based issues/discussions you can also try out Apache Druid's slack community : https://druid.apache.org/community/join-slack?v=1
NikitaElmanov
commented
2 years ago @rohangarg thanks for answer!
).
).NikitaElmanov
commented
2 years ago @rohangarg
Further, I think for setup based issues/discussions you can also try out Apache Druid's slack community : https://druid.apache.org/community/join-slack?v=1
Can not sing up with a mail. I suppose it will be better continue discussion here
rohangarg
commented
2 years ago @NikitaElmanov : Thanks for trying out the suggestions!
Did you get a chance to see the error coming in the broker logs when you add the extension to it? I just tried locally with 0.24.0-SNAPSHOT was able to get it to work. Adding it only in router wouldn't be sufficient since the query object is needed in broker for execution - as you observed with unsupported query type.
I'd like to know, can i use moving-average functionality in druid sql select query? Find nothing about it in docs: https://druid.apache.org/docs/latest/development/extensions-contrib/moving-average-query.html. There are only hard-to-read json examples.
No, there is no support in Druid SQL for moving-average query as of now. Only native json querying is supported. Do you think that SQL API will be useful for moving-average? If so, I think you can open a new issue requesting support for it.
Can not sing up with a mail. I suppose it will be better continue discussion here
Oh, I see - can you please share the error you see while joining if possible?
NikitaElmanov
commented
2 years ago @rohangarg)
Did you get a chance to see the error coming in the broker logs when you add the extension to it? I just tried locally with 0.24.0-SNAPSHOT was able to get it to work. Adding it only in router wouldn't be sufficient since the query object is needed in broker for execution - as you observed with
unsupported query type.
Tried to add druid.extensions.loadList=["druid-moving-average-query"] into /apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/broker/runtime.properties one more time and now do not get error 500, that i've had before. So strangely, It seems but you somehow influenced on it). There is no error in druid web, but exception with message unsupported query type stayed(.
log service druid in linux server: May 31 16:01:00 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:00 2022] Sending signal[15] to command[middleManager] (timeout 10s). May 31 16:01:00 postgresql systemd[1]: Stopping druid.service... May 31 16:01:00 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:00 2022] Command[zk] exited (pid = 26170, exited = 143) May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Sending signal[9] to command[coordinator-overlord]. May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Sending signal[9] to command[broker]. May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Sending signal[9] to command[router]. May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Sending signal[9] to command[historical]. May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Sending signal[9] to command[middleManager]. May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Command[middleManager] exited (pid = 26175, signal = 9) May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Command[router] exited (pid = 26173, signal = 9) May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Command[coordinator-overlord] exited (pid = 26171, signal = 9) May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Command[historical] exited (pid = 26174, signal = 9) May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Command[broker] exited (pid = 26172, signal = 9) May 31 16:01:11 postgresql start-micro-quickstart[26146]: [Tue May 31 16:01:11 2022] Exiting. May 31 16:02:41 postgresql systemd[1]: druid.service stop-final-sigterm timed out. Killing. May 31 16:02:41 postgresql systemd[1]: Unit druid.service entered failed state. May 31 16:02:41 postgresql systemd[1]: druid.service failed. May 31 16:02:41 postgresql systemd[1]: Started druid.service. May 31 16:02:42 postgresql start-micro-quickstart[28817]: [Tue May 31 16:02:42 2022] Running command[zk], logging to[/root/apache-druid-0.22.1/var/sv/zk.log]: bin/run-zk conf May 31 16:02:42 postgresql start-micro-quickstart[28817]: [Tue May 31 16:02:42 2022] Running command[coordinator-overlord], logging to[/root/apache-druid-0.22.1/var/sv/coordinator-overlord.log]: bin/run-druid coord May 31 16:02:42 postgresql start-micro-quickstart[28817]: [Tue May 31 16:02:42 2022] Running command[broker], logging to[/root/apache-druid-0.22.1/var/sv/broker.log]: bin/run-druid broker conf/druid/single-server/m May 31 16:02:42 postgresql start-micro-quickstart[28817]: [Tue May 31 16:02:42 2022] Running command[router], logging to[/root/apache-druid-0.22.1/var/sv/router.log]: bin/run-druid router conf/druid/single-server/m May 31 16:02:42 postgresql start-micro-quickstart[28817]: [Tue May 31 16:02:42 2022] Running command[historical], logging to[/root/apache-druid-0.22.1/var/sv/historical.log]: bin/run-druid historical conf/druid/sin May 31 16:02:42 postgresql start-micro-quickstart[28817]: [Tue May 31 16:02:42 2022] Running command[middleManager], logging to[/root/apache-druid-0.22.1/var/sv/middleManager.log]: bin/run-druid middleManager conf/
log /apache-druid-0.22.1/var/sv/broker.log:
_2022-05-31T13:02:46,744 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.2.5.Final
2022-05-31T13:02:48,758 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-moving-average-query], jars: druid-moving-average-query-0.22.1.jar
2022-05-31T13:02:49,433 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-moving-average-query], jars: druid-moving-average-query-0.22.1.jar
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.5.9-83df9301aa5c2a5d284a9940177808c01bc35cef, built on 01/06/2021 20:03 GMT
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:host.name=postgresql
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_302
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Red Hat, Inc.
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.302.b08-0.el7_9.x86_64/jre
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/root/apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/broker:/root/apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/_common:/root/apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/_common/hadoop-xml:/root/apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/../_common:/root/apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/../_common/hadoop-xml:/root/apache-druid-0.22.1/bin/../lib/netty-buffer-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/maven-repository-metadata-3.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/reactive-streams-1.0.2.jar:/root/apache-druid-0.22.1/bin/../lib/jersey-guice-1.19.3.jar:/root/apache-druid-0.22.1/bin/../lib/netty-transport-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-util-ajax-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/fastutil-core-8.5.4.jar:/root/apache-druid-0.22.1/bin/../lib/javax.el-api-3.0.0.jar:/root/apache-druid-0.22.1/bin/../lib/jvm-attach-api-1.5.jar:/root/apache-druid-0.22.1/bin/../lib/maven-aether-provider-3.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/druid-sql-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/javax.servlet-api-3.1.0.jar:/root/apache-druid-0.22.1/bin/../lib/resilience4j-bulkhead-1.3.1.jar:/root/apache-druid-0.22.1/bin/../lib/druid-console-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/aether-spi-0.9.0.M2.jar:/root/apache-druid-0.22.1/bin/../lib/aopalliance-1.0.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-rewrite-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/log4j-1.2-api-2.15.0.jar:/root/apache-druid-0.22.1/bin/../lib/avatica-server-1.17.0.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-jaxrs-json-provider-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/javax.el-3.0.0.jar:/root/apache-druid-0.22.1/bin/../lib/asm-commons-7.1.jar:/root/apache-druid-0.22.1/bin/../lib/datasketches-java-2.0.0.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-databind-2.10.5.1.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-jq-0.0.10.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-servlets-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/maven-settings-builder-3.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/plexus-interpolation-1.19.jar:/root/apache-druid-0.22.1/bin/../lib/ion-java-1.0.2.jar:/root/apache-druid-0.22.1/bin/../lib/resilience4j-core-1.3.1.jar:/root/apache-druid-0.22.1/bin/../lib/jaxb-api-2.3.1.jar:/root/apache-druid-0.22.1/bin/../lib/commons-collections-3.2.2.jar:/root/apache-druid-0.22.1/bin/../lib/plexus-utils-3.0.24.jar:/root/apache-druid-0.22.1/bin/../lib/joni-2.1.27.jar:/root/apache-druid-0.22.1/bin/../lib/icu4j-55.1.jar:/root/apache-druid-0.22.1/bin/../lib/log4j-slf4j-impl-2.15.0.jar:/root/apache-druid-0.22.1/bin/../lib/asm-tree-7.1.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-servlet-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/jsr305-2.0.1.jar:/root/apache-druid-0.22.1/bin/../lib/aether-connector-file-0.9.0.M2.jar:/root/apache-druid-0.22.1/bin/../lib/error_prone_annotations-2.8.0.jar:/root/apache-druid-0.22.1/bin/../lib/netty-resolver-dns-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/calcite-core-1.21.0.jar:/root/apache-druid-0.22.1/bin/../lib/netty-reactive-streams-2.0.0.jar:/root/apache-druid-0.22.1/bin/../lib/commons-logging-1.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/commons-compiler-3.0.11.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-io-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/jersey-core-1.19.3.jar:/root/apache-druid-0.22.1/bin/../lib/aggdesigner-algorithm-6.0.jar:/root/apache-druid-0.22.1/bin/../lib/jboss-logging-3.2.1.Final.jar:/root/apache-druid-0.22.1/bin/../lib/accessors-smart-1.2.jar:/root/apache-druid-0.22.1/bin/../lib/commons-lang3-3.8.1.jar:/root/apache-druid-0.22.1/bin/../lib/maven-artifact-3.6.0.jar:/root/apache-druid-0.22.1/bin/../lib/aws-java-sdk-core-1.12.37.jar:/root/apache-druid-0.22.1/bin/../lib/maven-model-3.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/jsr311-api-1.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/jersey-server-1.19.3.jar:/root/apache-druid-0.22.1/bin/../lib/guice-assistedinject-4.1.0.jar:/root/apache-druid-0.22.1/bin/../lib/jakarta.xml.bind-api-2.3.2.jar:/root/apache-druid-0.22.1/bin/../lib/aether-util-0.9.0.M2.jar:/root/apache-druid-0.22.1/bin/../lib/commons-collections4-4.2.jar:/root/apache-druid-0.22.1/bin/../lib/metrics-core-4.0.0.jar:/root/apache-druid-0.22.1/bin/../lib/extendedset-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/google-oauth-client-1.26.0.jar:/root/apache-druid-0.22.1/bin/../lib/httpcore-4.4.11.jar:/root/apache-druid-0.22.1/bin/../lib/aether-connector-okhttp-0.0.9.jar:/root/apache-druid-0.22.1/bin/../lib/netty-codec-socks-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-module-guice-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/zookeeper-3.5.9.jar:/root/apache-druid-0.22.1/bin/../lib/druid-processing-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/derbynet-10.14.2.0.jar:/root/apache-druid-0.22.1/bin/../lib/okhttp-1.0.2.jar:/root/apache-druid-0.22.1/bin/../lib/aws-java-sdk-s3-1.12.37.jar:/root/apache-druid-0.22.1/bin/../lib/joda-time-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/druid-aws-common-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/sigar-1.6.5.132.jar:/root/apache-druid-0.22.1/bin/../lib/netty-codec-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/fastutil-8.5.4.jar:/root/apache-druid-0.22.1/bin/../lib/google-http-client-1.26.0.jar:/root/apache-druid-0.22.1/bin/../lib/curator-x-discovery-4.3.0.jar:/root/apache-druid-0.22.1/bin/../lib/druid-services-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/netty-handler-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/druid-gcp-common-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/config-magic-0.9.jar:/root/apache-druid-0.22.1/bin/../lib/slf4j-api-1.7.25.jar:/root/apache-druid-0.22.1/bin/../lib/txw2-2.3.1.jar:/root/apache-druid-0.22.1/bin/../lib/log4j-core-2.15.0.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-security-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/javax.inject-1.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-dataformat-cbor-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-annotations-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-continuation-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/fastutil-extra-8.5.4.jar:/root/apache-druid-0.22.1/bin/../lib/guava-16.0.1.jar:/root/apache-druid-0.22.1/bin/../lib/istack-commons-runtime-3.0.7.jar:/root/apache-druid-0.22.1/bin/../lib/aether-api-0.9.0.M2.jar:/root/apache-druid-0.22.1/bin/../lib/commons-math3-3.6.1.jar:/root/apache-druid-0.22.1/bin/../lib/jdbi-2.63.1.jar:/root/apache-druid-0.22.1/bin/../lib/commons-beanutils-1.9.4.jar:/root/apache-druid-0.22.1/bin/../lib/rhino-1.7.11.jar:/root/apache-druid-0.22.1/bin/../lib/druid-moving-average-query-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/commons-lang-2.6.jar:/root/apache-druid-0.22.1/bin/../lib/commons-text-1.3.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-server-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/avatica-core-1.17.0.jar:/root/apache-druid-0.22.1/bin/../lib/hibernate-validator-5.2.5.Final.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-datatype-guava-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/druid-server-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/druid-indexing-service-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/commons-io-2.11.0.jar:/root/apache-druid-0.22.1/bin/../lib/guice-servlet-4.1.0.jar:/root/apache-druid-0.22.1/bin/../lib/netty-common-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/netty-codec-dns-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/audience-annotations-0.5.0.jar:/root/apache-druid-0.22.1/bin/../lib/tesla-aether-0.0.5.jar:/root/apache-druid-0.22.1/bin/../lib/zstd-jni-1.3.3-1.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-jaxrs-smile-provider-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/jcl-over-slf4j-1.7.12.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-module-jaxb-annotations-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/log4j-jul-2.15.0.jar:/root/apache-druid-0.22.1/bin/../lib/commons-codec-1.13.jar:/root/apache-druid-0.22.1/bin/../lib/vavr-match-0.10.2.jar:/root/apache-druid-0.22.1/bin/../lib/jakarta.activation-api-1.2.1.jar:/root/apache-druid-0.22.1/bin/../lib/opencsv-4.6.jar:/root/apache-druid-0.22.1/bin/../lib/datasketches-memory-1.3.0.jar:/root/apache-druid-0.22.1/bin/../lib/netty-resolver-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/checker-qual-2.5.7.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-core-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/zookeeper-jute-3.5.9.jar:/root/apache-druid-0.22.1/bin/../lib/calcite-linq4j-1.21.0.jar:/root/apache-druid-0.22.1/bin/../lib/guice-4.1.0.jar:/root/apache-druid-0.22.1/bin/../lib/classmate-1.1.0.jar:/root/apache-druid-0.22.1/bin/../lib/druid-hll-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/jersey-servlet-1.19.3.jar:/root/apache-druid-0.22.1/bin/../lib/javax.activation-1.2.0.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-jaxrs-base-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/curator-client-4.3.0.jar:/root/apache-druid-0.22.1/bin/../lib/antlr4-runtime-4.5.1.jar:/root/apache-druid-0.22.1/bin/../lib/netty-3.10.6.Final.jar:/root/apache-druid-0.22.1/bin/../lib/shims-0.9.0.jar:/root/apache-druid-0.22.1/bin/../lib/validation-api-1.1.0.Final.jar:/root/apache-druid-0.22.1/bin/../lib/netty-handler-proxy-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/derby-10.14.2.0.jar:/root/apache-druid-0.22.1/bin/../lib/curator-recipes-4.3.0.jar:/root/apache-druid-0.22.1/bin/../lib/caffeine-2.8.0.jar:/root/apache-druid-0.22.1/bin/../lib/RoaringBitmap-0.9.0.jar:/root/apache-druid-0.22.1/bin/../lib/vavr-0.10.2.jar:/root/apache-druid-0.22.1/bin/../lib/stax-ex-1.8.jar:/root/apache-druid-0.22.1/bin/../lib/google-http-client-jackson2-1.26.0.jar:/root/apache-druid-0.22.1/bin/../lib/FastInfoset-1.2.15.jar:/root/apache-druid-0.22.1/bin/../lib/maven-model-builder-3.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/aws-java-sdk-kms-1.12.37.jar:/root/apache-druid-0.22.1/bin/../lib/commons-net-3.6.jar:/root/apache-druid-0.22.1/bin/../lib/aws-java-sdk-ec2-1.12.37.jar:/root/apache-druid-0.22.1/bin/../lib/jaxb-runtime-2.3.1.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-proxy-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/google-api-client-1.26.0.jar:/root/apache-druid-0.22.1/bin/../lib/netty-transport-native-epoll-4.1.68.Final-linux-x86_64.jar:/root/apache-druid-0.22.1/bin/../lib/xz-1.8.jar:/root/apache-druid-0.22.1/bin/../lib/lz4-java-1.7.1.jar:/root/apache-druid-0.22.1/bin/../lib/spymemcached-2.12.3.jar:/root/apache-druid-0.22.1/bin/../lib/cron-scheduler-0.1.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-util-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/protobuf-java-3.11.0.jar:/root/apache-druid-0.22.1/bin/../lib/async-http-client-2.5.3.jar:/root/apache-druid-0.22.1/bin/../lib/guice-multibindings-4.1.0.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-client-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/json-smart-2.3.jar:/root/apache-druid-0.22.1/bin/../lib/wagon-provider-api-2.4.jar:/root/apache-druid-0.22.1/bin/../lib/druid-core-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/jmespath-java-1.12.37.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-dataformat-smile-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/httpclient-4.5.10.jar:/root/apache-druid-0.22.1/bin/../lib/airline-0.7.jar:/root/apache-druid-0.22.1/bin/../lib/druid-indexing-hadoop-0.22.1.jar:/root/apache-druid-0.22.1/bin/../lib/jetty-http-9.4.40.v20210413.jar:/root/apache-druid-0.22.1/bin/../lib/asm-analysis-7.1.jar:/root/apache-druid-0.22.1/bin/../lib/compress-lzf-1.0.4.jar:/root/apache-druid-0.22.1/bin/../lib/commons-compress-1.21.jar:/root/apache-druid-0.22.1/bin/../lib/maven-settings-3.1.1.jar:/root/apache-druid-0.22.1/bin/../lib/json-path-2.3.0.jar:/root/apache-druid-0.22.1/bin/../lib/aether-impl-0.9.0.M2.jar:/root/apache-druid-0.22.1/bin/../lib/esri-geometry-api-2.2.0.jar:/root/apache-druid-0.22.1/bin/../lib/avatica-metrics-1.17.0.jar:/root/apache-druid-0.22.1/bin/../lib/async-http-client-netty-utils-2.5.3.jar:/root/apache-druid-0.22.1/bin/../lib/curator-framework-4.3.0.jar:/root/apache-druid-0.22.1/bin/../lib/disruptor-3.3.6.jar:/root/apache-druid-0.22.1/bin/../lib/derbyclient-10.14.2.0.jar:/root/apache-druid-0.22.1/bin/../lib/netty-codec-http-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/j2objc-annotations-1.1.jar:/root/apache-druid-0.22.1/bin/../lib/asm-7.1.jar:/root/apache-druid-0.22.1/bin/../lib/commons-dbcp2-2.0.1.jar:/root/apache-druid-0.22.1/bin/../lib/jna-4.5.1.jar:/root/apache-druid-0.22.1/bin/../lib/janino-3.0.11.jar:/root/apache-druid-0.22.1/bin/../lib/commons-pool2-2.2.jar:/root/apache-druid-0.22.1/bin/../lib/netty-transport-native-unix-common-4.1.68.Final.jar:/root/apache-druid-0.22.1/bin/../lib/jcodings-1.0.43.jar:/root/apache-druid-0.22.1/bin/../lib/log4j-api-2.15.0.jar:/root/apache-druid-0.22.1/bin/../lib/jackson-datatype-joda-2.10.5.jar:/root/apache-druid-0.22.1/bin/../lib/javax.activation-api-1.2.0.jar
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=var/tmp
2022-05-31T13:02:54,449 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=
In my opinion, logs is correct, without errors and exceptions... Now, all setup is done and druid is working but "queryType": "movingAverage" is not recognized by druid(
rohangarg
commented
2 years ago @NikitaElmanov : Thanks for logs and the update - seems like the broker is configured correctly now as the logs also show loading of moving-average extension
2022-05-31T13:02:48,758 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-moving-average-query], jars: druid-moving-average-query-0.22.1.jar
Confirming whether you've also added this extension in router as well? Router needs it since it processes the query and adds some things for the broker and also chooses broker based on query. This process requires us to add moving-average jar in router as well.
One pointer to debug could be to look at the logs for broker and router when you fire the query to see where the log : "queryType": "movingAverage" is not recognized by druid( is coming from.
NikitaElmanov
commented
2 years ago Confirming whether you've also added this extension in router as well? Router needs it since it processes the query and adds some things for the broker and also chooses broker based on query. This process requires us to add moving-average jar in router as well.
@rohangarg: yes, confirming. Below is the content of /root/apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/broker/runtime.properties: /# /# Licensed to the Apache Software Foundation (ASF) under one /# or more contributor license agreements. See the NOTICE file /# distributed with this work for additional information /# regarding copyright ownership. The ASF licenses this file /# to you under the Apache License, Version 2.0 (the /# "License"); you may not use this file except in compliance /# with the License. You may obtain a copy of the License at /# /# http://www.apache.org/licenses/LICENSE-2.0 /# /# Unless required by applicable law or agreed to in writing, /# software distributed under the License is distributed on an /# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY /# KIND, either express or implied. See the License for the /# specific language governing permissions and limitations /# under the License. /# druid.extensions.loadList=["druid-moving-average-query"]
druid.service=druid/broker druid.plaintextPort=8082
/# HTTP server settings druid.server.http.numThreads=12
/# HTTP client settings druid.broker.http.numConnections=10 druid.broker.http.maxQueuedBytes=5MiB
/# Processing threads and buffers druid.processing.buffer.sizeBytes=100MiB druid.processing.numMergeBuffers=2 druid.processing.numThreads=1 druid.processing.tmpDir=var/druid/processing
/# Query cache disabled -- push down caching and merging instead druid.broker.cache.useCache=false druid.broker.cache.populateCache=false
Below is the content of /root/apache-druid-0.22.1/conf/druid/single-server/micro-quickstart/router/runtime.properties: /# /# Licensed to the Apache Software Foundation (ASF) under one /# or more contributor license agreements. See the NOTICE file /# distributed with this work for additional information /# regarding copyright ownership. The ASF licenses this file /# to you under the Apache License, Version 2.0 (the /# "License"); you may not use this file except in compliance /# with the License. You may obtain a copy of the License at /# /# http://www.apache.org/licenses/LICENSE-2.0 /# /# Unless required by applicable law or agreed to in writing, /# software distributed under the License is distributed on an /# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY /# KIND, either express or implied. See the License for the /# specific language governing permissions and limitations /# under the License. /# druid.extensions.loadList=["druid-moving-average-query"]
druid.service=druid/router druid.plaintextPort=8888
/# HTTP proxy druid.router.http.numConnections=50 druid.router.http.readTimeout=PT5M druid.router.http.numMaxThreads=100 druid.server.http.numThreads=100
/# Service discovery druid.router.defaultBrokerServiceName=druid/broker druid.router.coordinatorServiceName=druid/coordinator
/# Management proxy to coordinator / overlord: required for unified web console. druid.router.managementProxy.enabled=true
NikitaElmanov
commented
2 years ago Maybe add druid.extensions.loadList=["druid-moving-average-query"] in ./conf/druid/single-server/micro-quickstart/historical/runtime.properties ./conf/druid/single-server/micro-quickstart/coordinator-overlord/runtime.properties _./conf/druid/single-server/micro-quickstart/common/common.runtime.properties ./conf/druid/single-server/micro-quickstart/middleManager/runtime.properties or one in one of them?
rohangarg
commented
2 years ago Oh, I see. I locally tried two ways :
druid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches"] in conf/druid/single-server/micro-quickstart/_common/common.runtime.properties and druid.extensions.loadList=["druid-moving-average-query"] in broker and router's runtime.propertiesdruid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches", "druid-moving-average-query"] in conf/druid/single-server/micro-quickstart/_common/common.runtime.propertiesIn both the cases, the movingAverage query was working for me. Maybe, you can also try the 2) method once as well. Also, were you able to spot where the unsupported query type log comes in the druid processes?
NikitaElmanov
commented
2 years ago
- have,
druid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches"]inconf/druid/single-server/micro-quickstart/_common/common.runtime.propertiesanddruid.extensions.loadList=["druid-moving-average-query"]in broker and router's runtime.properties
It doesn't work for me. And 1) the method was implemented initially.
2. have,
druid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches", "druid-moving-average-query"]inconf/druid/single-server/micro-quickstart/_common/common.runtime.properties
does not work as well.
Also, have, druid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches", "druid-moving-average-query"] in conf/druid/single-server/micro-quickstart/_common/common.runtime.properties and druid.extensions.loadList=["druid-moving-average-query"] in broker and router's runtime.properties
does not work again(
In both the cases, the
movingAveragequery was working for me. Maybe, you can also try the 2) method once as well. Also, were you able to spot where theunsupported query typelog comes in the druid processes?
Did not find anything about 'unsupported query type' in logs. It seems to me pop-up messages do not put in logs. However you must know better) Maybe i am looking for in wrong place. Help me)
What druid version do you use?
rohangarg
commented
2 years ago I tried initially with 0.24.0-SNAPSHOT and it worked. I just tried with 0.22.1 version as well and it works for me. :) I did the following for 0.22.1 :
java -classpath "<deployment-root-path>/lib/*" org.apache.druid.cli.Main tools pull-deps -c org.apache.druid.extensions.contrib:druid-moving-average-query:0.22.1"druid-moving-average-query" for druid.extensions.loadList config in conf/druid/single-server/micro-quickstart/_common/common.runtime.properties. the config looks like : druid.extensions.loadList=["druid-hdfs-storage", "druid-kafka-indexing-service", "druid-datasketches", "druid-moving-average-query"]And tried the movingAverage query example from the documentation from druid console.
I'd suggest you to try these steps once locally and run a query from druid console to get to a running installation.
NikitaElmanov
commented
2 years ago Moving-average works in druid! Now I have a problem with a grafana) because it does not support movingAverage. I see that errors with grafana're not you problem but may be you faced with that one already?
kfaraz
commented
2 years ago @NikitaElmanov , please close this issue if the moving average works for you. For other troubleshooting which does not deal with Druid itself, you can seek assistance on the Druid slack.
I need to add moving average query for druid (version: 0.22.1). Following docs: https://druid.apache.org/docs/latest/development/extensions-contrib/moving-average-query.html use command: *java -classpath "/lib/ " org.apache.druid.cli.Main tools pull-deps -c org.apache.druid.extensions.contrib:druid-moving-average-query:0.22.1**
and get error message:
14:43:26.009 [main] ERROR org.apache.druid.cli.PullDependencies - Unable to resolve artifacts for [org.apache.druid.extensions.contrib:druid-moving-average-query:jar:0.22.1 (runtime) -> [] < [ (https://repo1.maven.org/maven2/, releases+snapshots)]]. java.lang.NullPointerException: null at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:361) ~[aether-impl-0.9.0.M2.jar:?] at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) ~[tesla-aether-0.0.5.jar:0.0.5] at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:407) [druid-services-0.22.1.jar:0.22.1] at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:365) [druid-services-0.22.1.jar:0.22.1] at org.apache.druid.cli.PullDependencies.run(PullDependencies.java:311) [druid-services-0.22.1.jar:0.22.1] at org.apache.druid.cli.Main.main(Main.java:113) [druid-services-0.22.1.jar:0.22.1] Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException at org.apache.druid.cli.PullDependencies.run(PullDependencies.java:335) at org.apache.druid.cli.Main.main(Main.java:113) Caused by: java.lang.RuntimeException: java.lang.NullPointerException at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:420) at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:365) at org.apache.druid.cli.PullDependencies.run(PullDependencies.java:311) ... 1 more Caused by: java.lang.NullPointerException at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:361) at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) at org.apache.druid.cli.PullDependencies.downloadExtension(PullDependencies.java:407) ... 3 more
In my opinion, my problem reason is linux server with druid placed in closed net. There's no access to internet and therefore to https://repo1.maven.org/maven2/. Please prompt me, how to setup druid with custom settings.xml file with my specific repositories?