search

apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.49k stars 3.7k forks source link

Servers not always showing up in the ui #8869

Closed ceastman-ibm closed 4 years ago

ceastman-ibm commented 4 years ago

Please provide a detailed title (e.g. "Broker crashes when using TopN query with Bound filter" instead of just "Broker crashes").

Affected Version

apache-druid-0.16.0-incubating

Description

Please include as much detailed information about the problem as possible.

sometimes when i reinstall my druid cluster in kubernetes it shows all the servers, other times it shows just the historical and middle manager. Where does this page get its information from? Is there a log i can look at to see why the query and master servers are not showing up?

kubectl get pods |grep druid druid-7fb8b697ff-vb76w 1/1 Running 0 29m druid-data-0 1/1 Running 0 29m druid-query-645f56d748-4q68p 1/1 Running 0 29m

Carls-MacBook-Pro:configs carleastman$ kubectl get svc |grep druid druid-broker ClusterIP 172.21.7.8 8082/TCP 29m druid-coordinator ClusterIP 172.21.68.222 8081/TCP 29m druid-historical ClusterIP 172.21.236.141 8083/TCP 29m druid-middlemanager ClusterIP 172.21.252.119 8091/TCP,8100/TCP,8101/TCP,8102/TCP,8103/TCP,8104/TCP,8105/TCP,8106/TCP,8107/TCP,8108/TCP,8109/TCP,8110/TCP,8111/TCP,8112/TCP,8113/TCP,8114/TCP,8115/TCP 29m druid-overlord ClusterIP 172.21.21.152 8090/TCP 29m druid-router ClusterIP 172.21.197.137 8888/TCP 29m druid-zookeeper ClusterIP 172.21.45.187 2181/TCP

Screen Shot 2019-11-14 at 3 03 40 PM
surekhasaharan commented 4 years ago

Where does this page get its information from?

This information is retrieved from sys.servers virtual table, which in turn uses DruidNodeDiscoveryProvider via zookeeper announcement

Is there a log i can look at to see why the query and master servers are not showing up?

May be zk logs or broker logs.

ceastman-ibm commented 4 years ago

@surekhasaharan so i looked in the druid-zookeeper and i only see one announcement for the server that is showing up. Do the announcements only happen once when the various servers start up? I looked in the other servers logs and it appears that they think they announced themselves successfully.

ceastman-ibm commented 4 years ago
Screen Shot 2019-11-15 at 8 08 11 AM Screen Shot 2019-11-15 at 8 08 47 AM
ceastman-ibm commented 4 years ago

currently zookeeper is running on the master node.

ceastman-ibm commented 4 years ago

here is the log from the broker:

root@druid-query-645f56d748-4q68p:/druid/var/sv# cat broker.log 2019-11-14T20:36:59,837 INFO [main] org.apache.druid.guice.PropertiesModule - Loading properties from common.runtime.properties 2019-11-14T20:36:59,845 INFO [main] org.apache.druid.guice.PropertiesModule - Loading properties from runtime.properties 2019-11-14T20:36:59,903 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final 2019-11-14T20:37:00,696 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.common.config.NullValueHandlingConfig] from props[druid.generic.] as [org.apache.druid.common.config.NullValueHandlingConfig@57abad67] 2019-11-14T20:37:00,789 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, directory='extensions', useExtensionClassloaderFirst=false, hadoopDependenciesDir='hadoop-dependencies', hadoopContainerDruidClasspath='null', addExtensionsToHadoopContainer=false, loadList=[druid-s3-extensions, druid-kafka-indexing-service, druid-datasketches]}] 2019-11-14T20:37:00,805 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-s3-extensions] for class [interface org.apache.druid.cli.CliCommandCreator] 2019-11-14T20:37:00,830 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-s3-extensions/druid-s3-extensions-0.16.0-incubating.jar] for extension[druid-s3-extensions] 2019-11-14T20:37:00,832 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service] for class [interface org.apache.druid.cli.CliCommandCreator] 2019-11-14T20:37:00,833 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-kafka-indexing-service/kafka-clients-2.1.0.jar] for extension[druid-kafka-indexing-service] 2019-11-14T20:37:00,833 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-kafka-indexing-service/druid-kafka-indexing-service-0.16.0-incubating.jar] for extension[druid-kafka-indexing-service] 2019-11-14T20:37:00,833 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-kafka-indexing-service/snappy-java-1.1.7.2.jar] for extension[druid-kafka-indexing-service] 2019-11-14T20:37:00,833 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-kafka-indexing-service/lz4-java-1.6.0.jar] for extension[druid-kafka-indexing-service] 2019-11-14T20:37:00,834 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-kafka-indexing-service/zstd-jni-1.3.3-1.jar] for extension[druid-kafka-indexing-service] 2019-11-14T20:37:00,837 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-datasketches] for class [interface org.apache.druid.cli.CliCommandCreator] 2019-11-14T20:37:00,837 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/commons-math3-3.6.1.jar] for extension[druid-datasketches] 2019-11-14T20:37:00,837 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/slf4j-api-1.7.25.jar] for extension[druid-datasketches] 2019-11-14T20:37:00,838 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/sketches-core-0.13.4.jar] for extension[druid-datasketches] 2019-11-14T20:37:00,838 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/druid-datasketches-0.16.0-incubating.jar] for extension[druid-datasketches] 2019-11-14T20:37:00,838 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/druid/extensions/druid-datasketches/memory-0.12.2.jar] for extension[druid-datasketches] 2019-11-14T20:37:00,980 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.ModulesConfig] from props[druid.modules.] as [ModulesConfig{excludeList=[]}] 2019-11-14T20:37:01,288 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.common.aws.AWSModule] for class [interface org.apache.druid.initialization.DruidModule] from classpath extension 2019-11-14T20:37:01,300 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.common.gcp.GcpModule] for class [interface org.apache.druid.initialization.DruidModule] from classpath extension 2019-11-14T20:37:01,303 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-s3-extensions] for class [interface org.apache.druid.initialization.DruidModule] 2019-11-14T20:37:01,320 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.storage.s3.S3StorageDruidModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,322 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.firehose.s3.S3FirehoseDruidModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,323 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-kafka-indexing-service] for class [interface org.apache.druid.initialization.DruidModule] 2019-11-14T20:37:01,327 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.indexing.kafka.KafkaIndexTaskModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,327 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-datasketches] for class [interface org.apache.druid.initialization.DruidModule] 2019-11-14T20:37:01,337 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.query.aggregation.datasketches.theta.SketchModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,338 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.query.aggregation.datasketches.theta.oldapi.OldApiSketchModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,341 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.query.aggregation.datasketches.quantiles.DoublesSketchModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,343 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.query.aggregation.datasketches.tuple.ArrayOfDoublesSketchModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,345 INFO [main] org.apache.druid.initialization.Initialization - Adding implementation [org.apache.druid.query.aggregation.datasketches.hll.HllSketchModule] for class [interface org.apache.druid.initialization.DruidModule] from local file system extension 2019-11-14T20:37:01,729 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.server.StatusResource 2019-11-14T20:37:01,946 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.server.BrokerQueryResource 2019-11-14T20:37:01,947 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.server.http.BrokerResource 2019-11-14T20:37:01,948 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.server.ClientInfoResource 2019-11-14T20:37:01,949 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.client.HttpServerInventoryViewResource 2019-11-14T20:37:01,957 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.query.lookup.LookupListeningResource 2019-11-14T20:37:01,958 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.query.lookup.LookupIntrospectionResource 2019-11-14T20:37:02,039 INFO [main] org.apache.druid.guice.Jerseys - Adding Jersey resource: org.apache.druid.sql.http.SqlResource 2019-11-14T20:37:02,930 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.DruidNode] from props[druid.] as [DruidNode{serviceName='druid/broker', host='172.30.33.175', bindOnHost=false, port=-1, plaintextPort=8082, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}] 2019-11-14T20:37:02,956 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.curator.CuratorConfig] from props[druid.zk.service.] as [org.apache.druid.curator.CuratorConfig@7f5eae0f] 2019-11-14T20:37:02,967 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.curator.ExhibitorConfig] from props[druid.exhibitor.service.] as [org.apache.druid.curator.ExhibitorConfig@3f985a86] 2019-11-14T20:37:03,071 INFO [main] org.apache.curator.utils.Compatibility - Running in ZooKeeper 3.4.x compatibility mode 2019-11-14T20:37:03,073 WARN [main] org.apache.curator.retry.ExponentialBackoffRetry - maxRetries too large (30). Pinning to 29 2019-11-14T20:37:03,169 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.initialization.ZkPathsConfig] from props[druid.zk.paths.] as [org.apache.druid.server.initialization.ZkPathsConfig@22e2266d] 2019-11-14T20:37:03,173 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.initialization.CuratorDiscoveryConfig] from props[druid.discovery.curator.] as [org.apache.druid.server.initialization.CuratorDiscoveryConfig@1bc776b7] 2019-11-14T20:37:03,438 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.server.security.Escalator] from props[druid.escalator.] as [NoopEscalator{}] 2019-11-14T20:37:03,452 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.security.AuthConfig] from props[druid.auth.] as [AuthConfig{authenticatorChain=null, authorizers=null, unsecuredPaths=[], allowUnauthenticatedHttpOptions=false}] 2019-11-14T20:37:03,480 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.metrics.DruidMonitorSchedulerConfig] from props[druid.monitoring.] as [org.apache.druid.server.metrics.DruidMonitorSchedulerConfig@4acc5dff] 2019-11-14T20:37:03,488 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.metrics.MonitorsConfig] from props[druid.monitoring.] as [MonitorsConfig{monitors=[class org.apache.druid.java.util.metrics.JvmMonitor]}] 2019-11-14T20:37:03,489 INFO [main] org.apache.druid.server.emitter.EmitterModule - Underlying emitter for ServiceEmitter: org.apache.druid.java.util.emitter.core.NoopEmitter@30ca0779 2019-11-14T20:37:03,489 INFO [main] org.apache.druid.server.emitter.EmitterModule - Extra service dimensions: {version=0.16.0-incubating} 2019-11-14T20:37:03,527 INFO [main] org.apache.druid.server.metrics.MetricsModule - Adding monitor[org.apache.druid.java.util.metrics.JvmMonitor@4012d5bc] 2019-11-14T20:37:03,529 INFO [main] org.apache.druid.server.metrics.MetricsModule - Adding monitor[org.apache.druid.query.ExecutorServiceMonitor@fe34b86] 2019-11-14T20:37:03,542 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.client.cache.CacheProvider] from props[druid.cache.] as [org.apache.druid.client.cache.CaffeineCacheProvider@25673087] 2019-11-14T20:37:04,312 INFO [main] org.apache.druid.server.metrics.MetricsModule - Adding monitor[org.apache.druid.client.cache.CacheMonitor@21a9a705] 2019-11-14T20:37:04,314 INFO [main] org.apache.druid.server.metrics.MetricsModule - Adding monitor[org.apache.druid.sql.avatica.AvaticaMonitor@7affc159] 2019-11-14T20:37:04,314 INFO [main] org.apache.druid.server.metrics.MetricsModule - Adding monitor[org.apache.druid.server.initialization.jetty.JettyServerModule$JettyMonitor@4eb9f2af] 2019-11-14T20:37:04,338 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.log.StartupLoggingConfig] from props[druid.startup.logging.] as [org.apache.druid.server.log.StartupLoggingConfig@19b206ee] 2019-11-14T20:37:04,342 INFO [main] org.apache.druid.cli.CliBroker - Starting up with processors[1], memory[12,455,444,480], maxMemory[12,455,444,480], directMemory[6,442,450,944]. 2019-11-14T20:37:04,354 INFO [main] org.apache.druid.cli.CliBroker - awt.toolkit: sun.awt.X11.XToolkit 2019-11-14T20:37:04,354 INFO [main] org.apache.druid.cli.CliBroker - druid.broker.cache.populateCache: false 2019-11-14T20:37:04,356 INFO [main] org.apache.druid.cli.CliBroker - druid.broker.cache.useCache: false 2019-11-14T20:37:04,356 INFO [main] org.apache.druid.cli.CliBroker - druid.broker.http.maxQueuedBytes: 10000000 2019-11-14T20:37:04,356 INFO [main] org.apache.druid.cli.CliBroker - druid.broker.http.numConnections: 50 2019-11-14T20:37:04,356 INFO [main] org.apache.druid.cli.CliBroker - druid.emitter: noop 2019-11-14T20:37:04,356 INFO [main] org.apache.druid.cli.CliBroker - druid.emitter.logging.logLevel: info 2019-11-14T20:37:04,356 INFO [main] org.apache.druid.cli.CliBroker - druid.extensions.loadList: ["druid-s3-extensions", "druid-kafka-indexing-service", "druid-datasketches"] 2019-11-14T20:37:04,356 INFO [main] org.apache.druid.cli.CliBroker - druid.host: 172.30.33.175 2019-11-14T20:37:04,357 INFO [main] org.apache.druid.cli.CliBroker - druid.indexer.logs.directory: var/druid/indexing-logs 2019-11-14T20:37:04,358 INFO [main] org.apache.druid.cli.CliBroker - druid.indexer.logs.type: file 2019-11-14T20:37:04,358 INFO [main] org.apache.druid.cli.CliBroker - druid.indexing.doubleStorage: double 2019-11-14T20:37:04,359 INFO [main] org.apache.druid.cli.CliBroker - druid.lookup.enableLookupSyncOnStartup: false 2019-11-14T20:37:04,359 INFO [main] org.apache.druid.cli.CliBroker - druid.metadata.storage.connector.connectURI: jdbc:derby://localhost:1527/var/druid/metadata.db;create=true 2019-11-14T20:37:04,361 INFO [main] org.apache.druid.cli.CliBroker - druid.metadata.storage.connector.host: localhost 2019-11-14T20:37:04,362 INFO [main] org.apache.druid.cli.CliBroker - druid.metadata.storage.connector.port: 1527 2019-11-14T20:37:04,362 INFO [main] org.apache.druid.cli.CliBroker - druid.metadata.storage.type: derby 2019-11-14T20:37:04,362 INFO [main] org.apache.druid.cli.CliBroker - druid.monitoring.monitors: ["org.apache.druid.java.util.metrics.JvmMonitor"] 2019-11-14T20:37:04,362 INFO [main] org.apache.druid.cli.CliBroker - druid.plaintextPort: 8082 2019-11-14T20:37:04,362 INFO [main] org.apache.druid.cli.CliBroker - druid.processing.buffer.sizeBytes: 500000000 2019-11-14T20:37:04,362 INFO [main] org.apache.druid.cli.CliBroker - druid.processing.numMergeBuffers: 6 2019-11-14T20:37:04,362 INFO [main] org.apache.druid.cli.CliBroker - druid.processing.numThreads: 1 2019-11-14T20:37:04,365 INFO [main] org.apache.druid.cli.CliBroker - druid.processing.tmpDir: var/druid/processing 2019-11-14T20:37:04,366 INFO [main] org.apache.druid.cli.CliBroker - druid.s3.accessKey: 2019-11-14T20:37:04,366 INFO [main] org.apache.druid.cli.CliBroker - druid.s3.endpoint.url: 2019-11-14T20:37:04,367 INFO [main] org.apache.druid.cli.CliBroker - druid.s3.secretKey: 2019-11-14T20:37:04,367 INFO [main] org.apache.druid.cli.CliBroker - druid.selectors.coordinator.serviceName: druid/coordinator 2019-11-14T20:37:04,367 INFO [main] org.apache.druid.cli.CliBroker - druid.selectors.indexing.serviceName: druid/overlord 2019-11-14T20:37:04,367 INFO [main] org.apache.druid.cli.CliBroker - druid.server.hiddenProperties: ["druid.s3.accessKey","druid.s3.secretKey","druid.metadata.storage.connector.password"] 2019-11-14T20:37:04,367 INFO [main] org.apache.druid.cli.CliBroker - druid.server.http.numThreads: 60 2019-11-14T20:37:04,367 INFO [main] org.apache.druid.cli.CliBroker - druid.service: druid/broker 2019-11-14T20:37:04,368 INFO [main] org.apache.druid.cli.CliBroker - druid.sql.enable: true 2019-11-14T20:37:04,368 INFO [main] org.apache.druid.cli.CliBroker - druid.startup.logging.logProperties: true 2019-11-14T20:37:04,368 INFO [main] org.apache.druid.cli.CliBroker - druid.storage.baseKey: druid/segments 2019-11-14T20:37:04,368 INFO [main] org.apache.druid.cli.CliBroker - druid.storage.bucket: 2019-11-14T20:37:04,368 INFO [main] org.apache.druid.cli.CliBroker - druid.storage.storageDirectory: var/druid/segments 2019-11-14T20:37:04,369 INFO [main] org.apache.druid.cli.CliBroker - druid.storage.type: s3 2019-11-14T20:37:04,369 INFO [main] org.apache.druid.cli.CliBroker - druid.zk.paths.base: /druid 2019-11-14T20:37:04,369 INFO [main] org.apache.druid.cli.CliBroker - druid.zk.service.host: druid-zookeeper 2019-11-14T20:37:04,369 INFO [main] org.apache.druid.cli.CliBroker - file.encoding: UTF-8 2019-11-14T20:37:04,369 INFO [main] org.apache.druid.cli.CliBroker - file.encoding.pkg: sun.io 2019-11-14T20:37:04,369 INFO [main] org.apache.druid.cli.CliBroker - file.separator: / 2019-11-14T20:37:04,369 INFO [main] org.apache.druid.cli.CliBroker - java.awt.graphicsenv: sun.awt.X11GraphicsEnvironment 2019-11-14T20:37:04,370 INFO [main] org.apache.druid.cli.CliBroker - java.awt.printerjob: sun.print.PSPrinterJob 2019-11-14T20:37:04,370 INFO [main] org.apache.druid.cli.CliBroker - * java.class.path: /druid/conf/druid/cluster/query/broker:/druid/conf/druid/cluster/query/_common:/druid/conf/druid/cluster/query/_common/hadoop-xml:/druid/conf/druid/cluster/query/../_common:/druid/conf/druid/cluster/query/../_common/hadoop-xml:/druid/bin/../lib/accessors-smart-1.2.jar:/druid/bin/../lib/jackson-core-2.6.7.jar:/druid/bin/../lib/google-oauth-client-1.22.0.jar:/druid/bin/../lib/log4j-core-2.5.jar:/druid/bin/../lib/jaxb-runtime-2.3.1.jar:/druid/bin/../lib/druid-aws-common-0.16.0-incubating.jar:/druid/bin/../lib/jackson-dataformat-cbor-2.6.7.jar:/druid/bin/../lib/jackson-jaxrs-json-provider-2.6.7.jar:/druid/bin/../lib/google-http-client-1.22.0.jar:/druid/bin/../lib/druid-server-0.16.0-incubating.jar:/druid/bin/../lib/maven-repository-metadata-3.1.1.jar:/druid/bin/../lib/json-path-2.3.0.jar:/druid/bin/../lib/reactive-streams-1.0.2.jar:/druid/bin/../lib/log4j-api-2.5.jar:/druid/bin/../lib/maven-settings-3.1.1.jar:/druid/bin/../lib/httpcore-4.4.11.jar:/druid/bin/../lib/commons-compress-1.18.jar:/druid/bin/../lib/commons-pool2-2.2.jar:/druid/bin/../lib/commons-net-3.6.jar:/druid/bin/../lib/netty-handler-proxy-4.1.30.Final.jar:/druid/bin/../lib/stax-ex-1.8.jar:/druid/bin/../lib/jetty-server-9.4.10.v20180503.jar:/druid/bin/../lib/asm-commons-7.1.jar:/druid/bin/../lib/fastutil-8.2.3.jar:/druid/bin/../lib/aws-java-sdk-s3-1.11.199.jar:/druid/bin/../lib/commons-dbcp2-2.0.1.jar:/druid/bin/../lib/javax.el-api-3.0.0.jar:/druid/bin/../lib/javax.el-3.0.0.jar:/druid/bin/../lib/google-http-client-jackson2-1.22.0.jar:/druid/bin/../lib/json-smart-2.3.jar:/druid/bin/../lib/jdbi-2.63.1.jar:/druid/bin/../lib/netty-codec-4.1.30.Final.jar:/druid/bin/../lib/derbyclient-10.14.2.0.jar:/druid/bin/../lib/jetty-servlet-9.4.10.v20180503.jar:/druid/bin/../lib/commons-io-2.6.jar:/druid/bin/../lib/curator-recipes-4.1.0.jar:/druid/bin/../lib/xz-1.8.jar:/druid/bin/../lib/netty-transport-4.1.30.Final.jar:/druid/bin/../lib/aopalliance-1.0.jar:/druid/bin/../lib/commons-lang3-3.8.1.jar:/druid/bin/../lib/jackson-core-asl-1.9.13.jar:/druid/bin/../lib/jersey-guice-1.19.3.jar:/druid/bin/../lib/log4j-1.2-api-2.5.jar:/druid/bin/../lib/validation-api-1.1.0.Final.jar:/druid/bin/../lib/asm-tree-7.1.jar:/druid/bin/../lib/zookeeper-3.4.14.jar:/druid/bin/../lib/guice-servlet-4.1.0.jar:/druid/bin/../lib/commons-math3-3.6.1.jar:/druid/bin/../lib/jersey-servlet-1.19.3.jar:/druid/bin/../lib/netty-resolver-4.1.30.Final.jar:/druid/bin/../lib/jackson-jaxrs-smile-provider-2.6.7.jar:/druid/bin/../lib/javax.activation-1.2.0.jar:/druid/bin/../lib/aether-connector-file-0.9.0.M2.jar:/druid/bin/../lib/async-http-client-netty-utils-2.5.3.jar:/druid/bin/../lib/async-http-client-2.5.3.jar:/druid/bin/../lib/asm-analysis-7.1.jar:/druid/bin/../lib/config-magic-0.9.jar:/druid/bin/../lib/tesla-aether-0.0.5.jar:/druid/bin/../lib/plexus-interpolation-1.19.jar:/druid/bin/../lib/javax.inject-1.jar:/druid/bin/../lib/curator-client-4.1.0.jar:/druid/bin/../lib/jackson-datatype-joda-2.6.7.jar:/druid/bin/../lib/commons-collections4-4.2.jar:/druid/bin/../lib/aggdesigner-algorithm-6.0.jar:/druid/bin/../lib/sigar-1.6.5.132.jar:/druid/bin/../lib/metrics-core-4.0.0.jar:/druid/bin/../lib/netty-codec-socks-4.1.30.Final.jar:/druid/bin/../lib/extendedset-0.16.0-incubating.jar:/druid/bin/../lib/maven-aether-provider-3.1.1.jar:/druid/bin/../lib/commons-collections-3.2.2.jar:/druid/bin/../lib/icu4j-54.1.1.jar:/druid/bin/../lib/aether-spi-0.9.0.M2.jar:/druid/bin/../lib/calcite-linq4j-1.17.0.jar:/druid/bin/../lib/jersey-server-1.19.3.jar:/druid/bin/../lib/jetty-continuation-9.4.10.v20180503.jar:/druid/bin/../lib/maven-artifact-3.6.0.jar:/druid/bin/../lib/jackson-databind-2.6.7.jar:/druid/bin/../lib/aws-java-sdk-core-1.11.199.jar:/druid/bin/../lib/jackson-module-guice-2.6.7.jar:/druid/bin/../lib/spymemcached-2.12.3.jar:/druid/bin/../lib/disruptor-3.3.6.jar:/druid/bin/../lib/FastInfoset-1.2.15.jar:/druid/bin/../lib/jaxb-api-2.3.1.jar:/druid/bin/../lib/netty-common-4.1.30.Final.jar:/druid/bin/../lib/netty-transport-native-unix-common-4.1.30.Final.jar:/druid/bin/../lib/jsr311-api-1.1.1.jar:/druid/bin/../lib/druid-gcp-common-0.16.0-incubating.jar:/druid/bin/../lib/jackson-module-jaxb-annotations-2.6.7.jar:/druid/bin/../lib/jackson-mapper-asl-1.9.13.jar:/druid/bin/../lib/classmate-1.0.0.jar:/druid/bin/../lib/log4j-jul-2.5.jar:/druid/bin/../lib/druid-core-0.16.0-incubating.jar:/druid/bin/../lib/druid-services-0.16.0-incubating.jar:/druid/bin/../lib/druid-hll-0.16.0-incubating.jar:/druid/bin/../lib/slf4j-api-1.7.25.jar:/druid/bin/../lib/druid-sql-0.16.0-incubating.jar:/druid/bin/../lib/jetty-http-9.4.10.v20180503.jar:/druid/bin/../lib/aether-impl-0.9.0.M2.jar:/druid/bin/../lib/okhttp-1.0.2.jar:/druid/bin/../lib/jackson-jq-0.0.10.jar:/druid/bin/../lib/jsr305-2.0.1.jar:/druid/bin/../lib/protobuf-java-3.1.0.jar:/druid/bin/../lib/derby-10.14.2.0.jar:/druid/bin/../lib/commons-text-1.3.jar:/druid/bin/../lib/jetty-util-9.4.10.v20180503.jar:/druid/bin/../lib/log4j-slf4j-impl-2.5.jar:/druid/bin/../lib/jna-4.5.1.jar:/druid/bin/../lib/jboss-logging-3.1.3.GA.jar:/druid/bin/../lib/ion-java-1.0.2.jar:/druid/bin/../lib/airline-0.7.jar:/druid/bin/../lib/jetty-security-9.4.10.v20180503.jar:/druid/bin/../lib/netty-transport-native-epoll-4.1.30.Final-linux-x86_64.jar:/druid/bin/../lib/netty-codec-dns-4.1.30.Final.jar:/druid/bin/../lib/druid-processing-0.16.0-incubating.jar:/druid/bin/../lib/jetty-client-9.4.10.v20180503.jar:/druid/bin/../lib/guice-multibindings-4.1.0.jar:/druid/bin/../lib/jersey-core-1.19.3.jar:/druid/bin/../lib/commons-logging-1.1.1.jar:/druid/bin/../lib/maven-settings-builder-3.1.1.jar:/druid/bin/../lib/guice-4.1.0.jar:/druid/bin/../lib/shims-0.8.6.jar:/druid/bin/../lib/netty-resolver-dns-4.1.30.Final.jar:/druid/bin/../lib/druid-indexing-service-0.16.0-incubating.jar:/druid/bin/../lib/httpclient-4.5.3.jar:/druid/bin/../lib/netty-buffer-4.1.30.Final.jar:/druid/bin/../lib/javax.servlet-api-3.1.0.jar:/druid/bin/../lib/opencsv-4.6.jar:/druid/bin/../lib/jackson-datatype-guava-2.6.7.jar:/druid/bin/../lib/jackson-dataformat-smile-2.6.7.jar:/druid/bin/../lib/avatica-server-1.12.0.jar:/druid/bin/../lib/curator-framework-4.1.0.jar:/druid/bin/../lib/esri-geometry-api-2.0.0.jar:/druid/bin/../lib/javax.activation-api-1.2.0.jar:/druid/bin/../lib/avatica-metrics-1.12.0.jar:/druid/bin/../lib/commons-lang-2.6.jar:/druid/bin/../lib/druid-console-0.16.0-incubating.jar:/druid/bin/../lib/druid-indexing-hadoop-0.16.0-incubating.jar:/druid/bin/../lib/netty-reactive-streams-2.0.0.jar:/druid/bin/../lib/netty-codec-http-4.1.30.Final.jar:/druid/bin/../lib/calcite-core-1.17.0.jar:/druid/bin/../lib/commons-codec-1.12.jar:/druid/bin/../lib/google-api-client-1.22.0.jar:/druid/bin/../lib/RoaringBitmap-0.8.6.jar:/druid/bin/../lib/avatica-core-1.12.0.jar:/druid/bin/../lib/netty-handler-4.1.30.Final.jar:/druid/bin/../lib/jcodings-1.0.43.jar:/druid/bin/../lib/asm-7.1.jar:/druid/bin/../lib/jcl-over-slf4j-1.7.12.jar:/druid/bin/../lib/jetty-proxy-9.4.10.v20180503.jar:/druid/bin/../lib/commons-beanutils-1.9.3.jar:/druid/bin/../lib/joda-time-2.10.2.jar:/druid/bin/../lib/jvm-attach-api-1.5.jar:/druid/bin/../lib/hibernate-validator-5.1.3.Final.jar:/druid/bin/../lib/netty-3.10.6.Final.jar:/druid/bin/../lib/derbynet-10.14.2.0.jar:/druid/bin/../lib/wagon-provider-api-2.4.jar:/druid/bin/../lib/jackson-annotations-2.6.7.jar:/druid/bin/../lib/lz4-java-1.6.0.jar:/druid/bin/../lib/jetty-servlets-9.4.10.v20180503.jar:/druid/bin/../lib/antlr4-runtime-4.5.1.jar:/druid/bin/../lib/jmespath-java-1.11.199.jar:/druid/bin/../lib/joni-2.1.27.jar:/druid/bin/../lib/maven-model-3.1.1.jar:/druid/bin/../lib/aws-java-sdk-kms-1.11.199.jar:/druid/bin/../lib/aws-java-sdk-ec2-1.11.199.jar:/druid/bin/../lib/plexus-utils-3.0.15.jar:/druid/bin/../lib/jline-0.9.94.jar:/druid/bin/../lib/checker-qual-2.5.7.jar:/druid/bin/../lib/zstd-jni-1.3.3-1.jar:/druid/bin/../lib/aether-util-0.9.0.M2.jar:/druid/bin/../lib/jackson-jaxrs-base-2.6.7.jar:/druid/bin/../lib/jetty-io-9.4.10.v20180503.jar:/druid/bin/../lib/aether-connector-okhttp-0.0.9.jar:/druid/bin/../lib/curator-x-discovery-4.1.0.jar:/druid/bin/../lib/error_prone_annotations-2.3.2.jar:/druid/bin/../lib/txw2-2.3.1.jar:/druid/bin/../lib/aether-api-0.9.0.M2.jar:/druid/bin/../lib/janino-2.7.6.jar:/druid/bin/../lib/commons-compiler-2.7.6.jar:/druid/bin/../lib/guava-16.0.1.jar:/druid/bin/../lib/audience-annotations-0.5.0.jar:/druid/bin/../lib/caffeine-2.5.5.jar:/druid/bin/../lib/maven-model-builder-3.1.1.jar:/druid/bin/../lib/rhino-1.7.11.jar:/druid/bin/../lib/compress-lzf-1.0.4.jar:/druid/bin/../lib/istack-commons-runtime-3.0.7.jar 2019-11-14T20:37:04,370 INFO [main] org.apache.druid.cli.CliBroker - java.class.version: 52.0 2019-11-14T20:37:04,370 INFO [main] org.apache.druid.cli.CliBroker - java.endorsed.dirs: /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed 2019-11-14T20:37:04,370 INFO [main] org.apache.druid.cli.CliBroker - java.ext.dirs: /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/java/packages/lib/ext 2019-11-14T20:37:04,371 INFO [main] org.apache.druid.cli.CliBroker - java.home: /usr/lib/jvm/java-8-openjdk-amd64/jre 2019-11-14T20:37:04,371 INFO [main] org.apache.druid.cli.CliBroker - java.io.tmpdir: var/tmp 2019-11-14T20:37:04,371 INFO [main] org.apache.druid.cli.CliBroker - java.library.path: /usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib 2019-11-14T20:37:04,371 INFO [main] org.apache.druid.cli.CliBroker - java.runtime.name: OpenJDK Runtime Environment 2019-11-14T20:37:04,371 INFO [main] org.apache.druid.cli.CliBroker - java.runtime.version: 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 2019-11-14T20:37:04,372 INFO [main] org.apache.druid.cli.CliBroker - java.specification.name: Java Platform API Specification 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.specification.vendor: Oracle Corporation 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.specification.version: 1.8 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.util.logging.manager: org.apache.logging.log4j.jul.LogManager 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.vendor: Private Build 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.vendor.url: http://java.oracle.com/ 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.vendor.url.bug: http://bugreport.sun.com/bugreport/ 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.version: 1.8.0_222 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.vm.info: mixed mode 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.vm.name: OpenJDK 64-Bit Server VM 2019-11-14T20:37:04,374 INFO [main] org.apache.druid.cli.CliBroker - java.vm.specification.name: Java Virtual Machine Specification 2019-11-14T20:37:04,375 INFO [main] org.apache.druid.cli.CliBroker - java.vm.specification.vendor: Oracle Corporation 2019-11-14T20:37:04,375 INFO [main] org.apache.druid.cli.CliBroker - java.vm.specification.version: 1.8 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - java.vm.vendor: Private Build 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - java.vm.version: 25.222-b10 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - line.separator:

2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - log4j.shutdownCallbackRegistry: org.apache.druid.common.config.Log4jShutdown 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - log4j.shutdownHookEnabled: true 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - os.arch: amd64 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - os.name: Linux 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - os.version: 4.15.0-66-generic 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - path.separator: : 2019-11-14T20:37:04,376 INFO [main] org.apache.druid.cli.CliBroker - sun.arch.data.model: 64 2019-11-14T20:37:04,377 INFO [main] org.apache.druid.cli.CliBroker - sun.boot.class.path: /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jfr.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/classes 2019-11-14T20:37:04,377 INFO [main] org.apache.druid.cli.CliBroker - sun.boot.library.path: /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64 2019-11-14T20:37:04,377 INFO [main] org.apache.druid.cli.CliBroker - sun.cpu.endian: little 2019-11-14T20:37:04,377 INFO [main] org.apache.druid.cli.CliBroker - sun.cpu.isalist: 2019-11-14T20:37:04,377 INFO [main] org.apache.druid.cli.CliBroker - sun.io.unicode.encoding: UnicodeLittle 2019-11-14T20:37:04,377 INFO [main] org.apache.druid.cli.CliBroker - sun.java.command: org.apache.druid.cli.Main server broker 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - sun.java.launcher: SUN_STANDARD 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - sun.jnu.encoding: ANSI_X3.4-1968 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - sun.management.compiler: HotSpot 64-Bit Tiered Compilers 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - sun.os.patch.level: unknown 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - user.country: US 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - user.dir: /druid 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - user.home: /root 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - user.language: en 2019-11-14T20:37:04,378 INFO [main] org.apache.druid.cli.CliBroker - user.name: root 2019-11-14T20:37:04,379 INFO [main] org.apache.druid.cli.CliBroker - * user.timezone: UTC 2019-11-14T20:37:04,390 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [druid.computation.buffer.poolCacheMaxCount, ${base_path}.buffer.poolCacheMaxCount] on [org.apache.druid.query.DruidProcessingConfig#poolCacheMaxCount()] 2019-11-14T20:37:04,393 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [500000000] for [druid.processing.buffer.sizeBytes] on [org.apache.druid.query.DruidProcessingConfig#intermediateComputeSizeBytesConfigured()] 2019-11-14T20:37:04,394 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [6] for [druid.processing.numMergeBuffers] on [org.apache.druid.query.DruidProcessingConfig#getNumMergeBuffersConfigured()] 2019-11-14T20:37:04,395 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.columnCache.sizeBytes] on [org.apache.druid.query.DruidProcessingConfig#columnCacheSizeBytes()] 2019-11-14T20:37:04,395 INFO [main] org.skife.config.ConfigurationObjectFactory - Using method itself for [${base_path}.fifo] on [org.apache.druid.query.DruidProcessingConfig#isFifo()] 2019-11-14T20:37:04,395 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [1] for [druid.processing.numThreads] on [org.apache.druid.query.DruidProcessingConfig#getNumThreadsConfigured()] 2019-11-14T20:37:04,396 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning value [var/druid/processing] for [druid.processing.tmpDir] on [org.apache.druid.query.DruidProcessingConfig#getTmpDir()] 2019-11-14T20:37:04,396 INFO [main] org.skife.config.ConfigurationObjectFactory - Assigning default value [processing-%s] for [${base_path}.formatString] on [org.apache.druid.java.util.common.concurrent.ExecutorServiceConfig#getFormatString()] 2019-11-14T20:37:04,564 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.search.SearchQueryConfig] from props[druid.query.search.] as [org.apache.druid.query.search.SearchQueryConfig@50a13c2f] 2019-11-14T20:37:04,571 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.metadata.SegmentMetadataQueryConfig] from props[druid.query.segmentMetadata.] as [org.apache.druid.query.metadata.SegmentMetadataQueryConfig@1a07bf6] 2019-11-14T20:37:04,579 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.groupby.GroupByQueryConfig] from props[druid.query.groupBy.] as [GroupByQueryConfig{defaultStrategy='v2', singleThreaded=false, maxIntermediateRows=50000, maxResults=500000, bufferGrouperMaxSize=2147483647, bufferGrouperMaxLoadFactor=0.0, bufferGrouperInitialBuckets=0, maxMergingDictionarySize=100000000, maxOnDiskStorage=0, forcePushDownLimit=false, forceHashAggregation=false, intermediateCombineDegree=8, numParallelCombineThreads=1, vectorize=false, forcePushDownNestedQuery=false}] 2019-11-14T20:37:04,601 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new intermediate processing buffer[0] of size[500,000,000] 2019-11-14T20:37:04,993 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[0] of size[500,000,000] 2019-11-14T20:37:05,431 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[1] of size[500,000,000] 2019-11-14T20:37:05,837 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[2] of size[500,000,000] 2019-11-14T20:37:06,252 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[3] of size[500,000,000] 2019-11-14T20:37:06,638 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[4] of size[500,000,000] 2019-11-14T20:37:07,005 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[5] of size[500,000,000] 2019-11-14T20:37:07,373 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.scan.ScanQueryConfig] from props[druid.query.scan.] as [ScanQueryConfig{legacy=false}] 2019-11-14T20:37:07,376 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.topn.TopNQueryConfig] from props[druid.query.topN.] as [org.apache.druid.query.topn.TopNQueryConfig@78fe204a] 2019-11-14T20:37:07,383 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.http.DruidHttpClientConfig] from props[druid.broker.http.] as [org.apache.druid.guice.http.DruidHttpClientConfig@4373f66f] 2019-11-14T20:37:07,673 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.client.FilteredServerInventoryViewProvider] from props[druid.serverview.] as [org.apache.druid.client.FilteredBatchServerInventoryViewProvider@51fc862e] 2019-11-14T20:37:07,711 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.client.selector.ServerSelectorStrategy] from props[druid.broker.balancer.] as [org.apache.druid.client.selector.RandomServerSelectorStrategy@73fbdf68] 2019-11-14T20:37:07,714 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.client.selector.TierSelectorStrategy] from props[druid.broker.select.] as [org.apache.druid.client.selector.HighestPriorityTierSelectorStrategy@32f32623] 2019-11-14T20:37:07,717 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.client.BrokerSegmentWatcherConfig] from props[druid.broker.segment.] as [org.apache.druid.client.BrokerSegmentWatcherConfig@61db86bf] 2019-11-14T20:37:07,726 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.client.cache.CacheConfig] from props[druid.broker.cache.] as [org.apache.druid.client.cache.CacheConfig@6418e39e] 2019-11-14T20:37:07,730 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.http.DruidHttpClientConfig] from props[druid.broker.http.] as [org.apache.druid.guice.http.DruidHttpClientConfig@6b1b19cf] 2019-11-14T20:37:07,733 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.RetryQueryRunnerConfig] from props[druid.broker.retryPolicy.] as [org.apache.druid.query.RetryQueryRunnerConfig@33bb3f86] 2019-11-14T20:37:07,752 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.initialization.ServerConfig] from props[druid.server.http.] as [ServerConfig{numThreads=60, queueSize=2147483647, enableRequestLimit=false, maxIdleTime=PT5M, defaultQueryTimeout=300000, maxScatterGatherBytes=9223372036854775807, maxQueryTimeout=9223372036854775807, maxRequestHeaderSize=8192, gracefulShutdownTimeout=PT0S, unannouncePropagationDelay=PT0S, inflateBufferSize=4096, compressionLevel=-1}] 2019-11-14T20:37:07,756 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.server.log.RequestLoggerProvider] from props[druid.request.logging.] as [org.apache.druid.server.log.NoopRequestLoggerProvider@4a1a256d] 2019-11-14T20:37:07,767 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.server.initialization.TLSServerConfig] from props[druid.server.https.] as [TLSServerConfig{keyStorePath='null', keyStoreType='null', certAlias='null', keyManagerFactoryAlgorithm='null', includeCipherSuites=null, excludeCipherSuites=null, includeProtocols=null, excludeProtocols=null, requireClientCertificate=false, requestClientCertificate=false, trustStoreType='null', trustStorePath='null', trustStoreAlgorithm='null', validateHostnames='true', crlPath='null'}] 2019-11-14T20:37:07,781 INFO [main] org.eclipse.jetty.util.log - Logging initialized @9225ms to org.eclipse.jetty.util.log.Slf4jLog 2019-11-14T20:37:07,797 INFO [main] org.apache.druid.server.initialization.jetty.JettyServerModule - Creating http connector with port [8082] 2019-11-14T20:37:07,868 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.sql.calcite.planner.PlannerConfig] from props[druid.sql.planner.] as [PlannerConfig{metadataRefreshPeriod=PT1M, maxSemiJoinRowsInMemory=100000, maxTopNLimit=100000, maxQueryCount=8, useApproximateCountDistinct=true, useApproximateTopN=true, requireTimeCondition=false, awaitInitializationOnStart=true, metadataSegmentCacheEnable=false, metadataSegmentPollPeriod=60000, sqlTimeZone=UTC, serializeComplexValues=true}] 2019-11-14T20:37:07,875 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.guice.http.DruidHttpClientConfig] from props[druid.global.http.] as [org.apache.druid.guice.http.DruidHttpClientConfig@64b0d1fa] 2019-11-14T20:37:07,957 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.client.coordinator.CoordinatorSelectorConfig] from props[druid.selectors.coordinator.] as [org.apache.druid.client.coordinator.CoordinatorSelectorConfig@729c8063] 2019-11-14T20:37:08,004 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[interface org.apache.druid.client.ServerInventoryViewProvider] from props[druid.serverview.] as [org.apache.druid.client.BatchServerInventoryViewProvider@28babeca] 2019-11-14T20:37:08,009 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.client.indexing.IndexingServiceSelectorConfig] from props[druid.selectors.indexing.] as [org.apache.druid.client.indexing.IndexingServiceSelectorConfig@16f0ec18] 2019-11-14T20:37:08,151 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.lookup.LookupConfig] from props[druid.lookup.] as [LookupConfig{snapshotWorkingDir='', enableLookupSyncOnStartup=false, numLookupLoadingThreads=1, coordinatorFetchRetries=3, lookupStartRetries=3, coordinatorRetryDelay=60000}] 2019-11-14T20:37:08,155 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.query.lookup.LookupListeningAnnouncerConfig] from props[druid.lookup.] as [ListeningAnnouncerConfig{listenersPath='/druid/listeners'}] 2019-11-14T20:37:08,457 INFO [main] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.sql.avatica.AvaticaServerConfig] from props[druid.sql.avatica.] as [org.apache.druid.sql.avatica.AvaticaServerConfig@1eca3ea7] 2019-11-14T20:37:08,718 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [INIT] 2019-11-14T20:37:08,719 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.start()] on object[org.apache.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@34c62fdf]. 2019-11-14T20:37:08,719 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [NORMAL] 2019-11-14T20:37:08,719 INFO [main] org.apache.druid.curator.CuratorModule - Starting Curator 2019-11-14T20:37:08,719 INFO [main] org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:host.name=druid-query-645f56d748-4q68p 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_222 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Private Build 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/druid/conf/druid/cluster/query/broker:/druid/conf/druid/cluster/query/_common:/druid/conf/druid/cluster/query/_common/hadoop-xml:/druid/conf/druid/cluster/query/../_common:/druid/conf/druid/cluster/query/../_common/hadoop-xml:/druid/bin/../lib/accessors-smart-1.2.jar:/druid/bin/../lib/jackson-core-2.6.7.jar:/druid/bin/../lib/google-oauth-client-1.22.0.jar:/druid/bin/../lib/log4j-core-2.5.jar:/druid/bin/../lib/jaxb-runtime-2.3.1.jar:/druid/bin/../lib/druid-aws-common-0.16.0-incubating.jar:/druid/bin/../lib/jackson-dataformat-cbor-2.6.7.jar:/druid/bin/../lib/jackson-jaxrs-json-provider-2.6.7.jar:/druid/bin/../lib/google-http-client-1.22.0.jar:/druid/bin/../lib/druid-server-0.16.0-incubating.jar:/druid/bin/../lib/maven-repository-metadata-3.1.1.jar:/druid/bin/../lib/json-path-2.3.0.jar:/druid/bin/../lib/reactive-streams-1.0.2.jar:/druid/bin/../lib/log4j-api-2.5.jar:/druid/bin/../lib/maven-settings-3.1.1.jar:/druid/bin/../lib/httpcore-4.4.11.jar:/druid/bin/../lib/commons-compress-1.18.jar:/druid/bin/../lib/commons-pool2-2.2.jar:/druid/bin/../lib/commons-net-3.6.jar:/druid/bin/../lib/netty-handler-proxy-4.1.30.Final.jar:/druid/bin/../lib/stax-ex-1.8.jar:/druid/bin/../lib/jetty-server-9.4.10.v20180503.jar:/druid/bin/../lib/asm-commons-7.1.jar:/druid/bin/../lib/fastutil-8.2.3.jar:/druid/bin/../lib/aws-java-sdk-s3-1.11.199.jar:/druid/bin/../lib/commons-dbcp2-2.0.1.jar:/druid/bin/../lib/javax.el-api-3.0.0.jar:/druid/bin/../lib/javax.el-3.0.0.jar:/druid/bin/../lib/google-http-client-jackson2-1.22.0.jar:/druid/bin/../lib/json-smart-2.3.jar:/druid/bin/../lib/jdbi-2.63.1.jar:/druid/bin/../lib/netty-codec-4.1.30.Final.jar:/druid/bin/../lib/derbyclient-10.14.2.0.jar:/druid/bin/../lib/jetty-servlet-9.4.10.v20180503.jar:/druid/bin/../lib/commons-io-2.6.jar:/druid/bin/../lib/curator-recipes-4.1.0.jar:/druid/bin/../lib/xz-1.8.jar:/druid/bin/../lib/netty-transport-4.1.30.Final.jar:/druid/bin/../lib/aopalliance-1.0.jar:/druid/bin/../lib/commons-lang3-3.8.1.jar:/druid/bin/../lib/jackson-core-asl-1.9.13.jar:/druid/bin/../lib/jersey-guice-1.19.3.jar:/druid/bin/../lib/log4j-1.2-api-2.5.jar:/druid/bin/../lib/validation-api-1.1.0.Final.jar:/druid/bin/../lib/asm-tree-7.1.jar:/druid/bin/../lib/zookeeper-3.4.14.jar:/druid/bin/../lib/guice-servlet-4.1.0.jar:/druid/bin/../lib/commons-math3-3.6.1.jar:/druid/bin/../lib/jersey-servlet-1.19.3.jar:/druid/bin/../lib/netty-resolver-4.1.30.Final.jar:/druid/bin/../lib/jackson-jaxrs-smile-provider-2.6.7.jar:/druid/bin/../lib/javax.activation-1.2.0.jar:/druid/bin/../lib/aether-connector-file-0.9.0.M2.jar:/druid/bin/../lib/async-http-client-netty-utils-2.5.3.jar:/druid/bin/../lib/async-http-client-2.5.3.jar:/druid/bin/../lib/asm-analysis-7.1.jar:/druid/bin/../lib/config-magic-0.9.jar:/druid/bin/../lib/tesla-aether-0.0.5.jar:/druid/bin/../lib/plexus-interpolation-1.19.jar:/druid/bin/../lib/javax.inject-1.jar:/druid/bin/../lib/curator-client-4.1.0.jar:/druid/bin/../lib/jackson-datatype-joda-2.6.7.jar:/druid/bin/../lib/commons-collections4-4.2.jar:/druid/bin/../lib/aggdesigner-algorithm-6.0.jar:/druid/bin/../lib/sigar-1.6.5.132.jar:/druid/bin/../lib/metrics-core-4.0.0.jar:/druid/bin/../lib/netty-codec-socks-4.1.30.Final.jar:/druid/bin/../lib/extendedset-0.16.0-incubating.jar:/druid/bin/../lib/maven-aether-provider-3.1.1.jar:/druid/bin/../lib/commons-collections-3.2.2.jar:/druid/bin/../lib/icu4j-54.1.1.jar:/druid/bin/../lib/aether-spi-0.9.0.M2.jar:/druid/bin/../lib/calcite-linq4j-1.17.0.jar:/druid/bin/../lib/jersey-server-1.19.3.jar:/druid/bin/../lib/jetty-continuation-9.4.10.v20180503.jar:/druid/bin/../lib/maven-artifact-3.6.0.jar:/druid/bin/../lib/jackson-databind-2.6.7.jar:/druid/bin/../lib/aws-java-sdk-core-1.11.199.jar:/druid/bin/../lib/jackson-module-guice-2.6.7.jar:/druid/bin/../lib/spymemcached-2.12.3.jar:/druid/bin/../lib/disruptor-3.3.6.jar:/druid/bin/../lib/FastInfoset-1.2.15.jar:/druid/bin/../lib/jaxb-api-2.3.1.jar:/druid/bin/../lib/netty-common-4.1.30.Final.jar:/druid/bin/../lib/netty-transport-native-unix-common-4.1.30.Final.jar:/druid/bin/../lib/jsr311-api-1.1.1.jar:/druid/bin/../lib/druid-gcp-common-0.16.0-incubating.jar:/druid/bin/../lib/jackson-module-jaxb-annotations-2.6.7.jar:/druid/bin/../lib/jackson-mapper-asl-1.9.13.jar:/druid/bin/../lib/classmate-1.0.0.jar:/druid/bin/../lib/log4j-jul-2.5.jar:/druid/bin/../lib/druid-core-0.16.0-incubating.jar:/druid/bin/../lib/druid-services-0.16.0-incubating.jar:/druid/bin/../lib/druid-hll-0.16.0-incubating.jar:/druid/bin/../lib/slf4j-api-1.7.25.jar:/druid/bin/../lib/druid-sql-0.16.0-incubating.jar:/druid/bin/../lib/jetty-http-9.4.10.v20180503.jar:/druid/bin/../lib/aether-impl-0.9.0.M2.jar:/druid/bin/../lib/okhttp-1.0.2.jar:/druid/bin/../lib/jackson-jq-0.0.10.jar:/druid/bin/../lib/jsr305-2.0.1.jar:/druid/bin/../lib/protobuf-java-3.1.0.jar:/druid/bin/../lib/derby-10.14.2.0.jar:/druid/bin/../lib/commons-text-1.3.jar:/druid/bin/../lib/jetty-util-9.4.10.v20180503.jar:/druid/bin/../lib/log4j-slf4j-impl-2.5.jar:/druid/bin/../lib/jna-4.5.1.jar:/druid/bin/../lib/jboss-logging-3.1.3.GA.jar:/druid/bin/../lib/ion-java-1.0.2.jar:/druid/bin/../lib/airline-0.7.jar:/druid/bin/../lib/jetty-security-9.4.10.v20180503.jar:/druid/bin/../lib/netty-transport-native-epoll-4.1.30.Final-linux-x86_64.jar:/druid/bin/../lib/netty-codec-dns-4.1.30.Final.jar:/druid/bin/../lib/druid-processing-0.16.0-incubating.jar:/druid/bin/../lib/jetty-client-9.4.10.v20180503.jar:/druid/bin/../lib/guice-multibindings-4.1.0.jar:/druid/bin/../lib/jersey-core-1.19.3.jar:/druid/bin/../lib/commons-logging-1.1.1.jar:/druid/bin/../lib/maven-settings-builder-3.1.1.jar:/druid/bin/../lib/guice-4.1.0.jar:/druid/bin/../lib/shims-0.8.6.jar:/druid/bin/../lib/netty-resolver-dns-4.1.30.Final.jar:/druid/bin/../lib/druid-indexing-service-0.16.0-incubating.jar:/druid/bin/../lib/httpclient-4.5.3.jar:/druid/bin/../lib/netty-buffer-4.1.30.Final.jar:/druid/bin/../lib/javax.servlet-api-3.1.0.jar:/druid/bin/../lib/opencsv-4.6.jar:/druid/bin/../lib/jackson-datatype-guava-2.6.7.jar:/druid/bin/../lib/jackson-dataformat-smile-2.6.7.jar:/druid/bin/../lib/avatica-server-1.12.0.jar:/druid/bin/../lib/curator-framework-4.1.0.jar:/druid/bin/../lib/esri-geometry-api-2.0.0.jar:/druid/bin/../lib/javax.activation-api-1.2.0.jar:/druid/bin/../lib/avatica-metrics-1.12.0.jar:/druid/bin/../lib/commons-lang-2.6.jar:/druid/bin/../lib/druid-console-0.16.0-incubating.jar:/druid/bin/../lib/druid-indexing-hadoop-0.16.0-incubating.jar:/druid/bin/../lib/netty-reactive-streams-2.0.0.jar:/druid/bin/../lib/netty-codec-http-4.1.30.Final.jar:/druid/bin/../lib/calcite-core-1.17.0.jar:/druid/bin/../lib/commons-codec-1.12.jar:/druid/bin/../lib/google-api-client-1.22.0.jar:/druid/bin/../lib/RoaringBitmap-0.8.6.jar:/druid/bin/../lib/avatica-core-1.12.0.jar:/druid/bin/../lib/netty-handler-4.1.30.Final.jar:/druid/bin/../lib/jcodings-1.0.43.jar:/druid/bin/../lib/asm-7.1.jar:/druid/bin/../lib/jcl-over-slf4j-1.7.12.jar:/druid/bin/../lib/jetty-proxy-9.4.10.v20180503.jar:/druid/bin/../lib/commons-beanutils-1.9.3.jar:/druid/bin/../lib/joda-time-2.10.2.jar:/druid/bin/../lib/jvm-attach-api-1.5.jar:/druid/bin/../lib/hibernate-validator-5.1.3.Final.jar:/druid/bin/../lib/netty-3.10.6.Final.jar:/druid/bin/../lib/derbynet-10.14.2.0.jar:/druid/bin/../lib/wagon-provider-api-2.4.jar:/druid/bin/../lib/jackson-annotations-2.6.7.jar:/druid/bin/../lib/lz4-java-1.6.0.jar:/druid/bin/../lib/jetty-servlets-9.4.10.v20180503.jar:/druid/bin/../lib/antlr4-runtime-4.5.1.jar:/druid/bin/../lib/jmespath-java-1.11.199.jar:/druid/bin/../lib/joni-2.1.27.jar:/druid/bin/../lib/maven-model-3.1.1.jar:/druid/bin/../lib/aws-java-sdk-kms-1.11.199.jar:/druid/bin/../lib/aws-java-sdk-ec2-1.11.199.jar:/druid/bin/../lib/plexus-utils-3.0.15.jar:/druid/bin/../lib/jline-0.9.94.jar:/druid/bin/../lib/checker-qual-2.5.7.jar:/druid/bin/../lib/zstd-jni-1.3.3-1.jar:/druid/bin/../lib/aether-util-0.9.0.M2.jar:/druid/bin/../lib/jackson-jaxrs-base-2.6.7.jar:/druid/bin/../lib/jetty-io-9.4.10.v20180503.jar:/druid/bin/../lib/aether-connector-okhttp-0.0.9.jar:/druid/bin/../lib/curator-x-discovery-4.1.0.jar:/druid/bin/../lib/error_prone_annotations-2.3.2.jar:/druid/bin/../lib/txw2-2.3.1.jar:/druid/bin/../lib/aether-api-0.9.0.M2.jar:/druid/bin/../lib/janino-2.7.6.jar:/druid/bin/../lib/commons-compiler-2.7.6.jar:/druid/bin/../lib/guava-16.0.1.jar:/druid/bin/../lib/audience-annotations-0.5.0.jar:/druid/bin/../lib/caffeine-2.5.5.jar:/druid/bin/../lib/maven-model-builder-3.1.1.jar:/druid/bin/../lib/rhino-1.7.11.jar:/druid/bin/../lib/compress-lzf-1.0.4.jar:/druid/bin/../lib/istack-commons-runtime-3.0.7.jar 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=var/tmp 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:java.compiler= 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:os.version=4.15.0-66-generic 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.name=root 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.home=/root 2019-11-14T20:37:08,730 INFO [main] org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/druid 2019-11-14T20:37:08,731 INFO [main] org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=druid-zookeeper sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@2f09e6b2 2019-11-14T20:37:08,774 INFO [main] org.apache.curator.framework.imps.CuratorFrameworkImpl - Default schema 2019-11-14T20:37:08,775 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.java.util.emitter.service.ServiceEmitter.start()] on object[ServiceEmitter{serviceDimensions={service=druid/broker, host=172.30.33.175:8082, version=0.16.0-incubating}, emitter=org.apache.druid.java.util.emitter.core.NoopEmitter@30ca0779}]. 2019-11-14T20:37:08,776 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.java.util.metrics.MonitorScheduler.start()] on object[org.apache.druid.java.util.metrics.MonitorScheduler@5866731]. 2019-11-14T20:37:08,807 INFO [main-SendThread(druid-zookeeper:2181)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server druid-zookeeper/172.21.45.187:2181. Will not attempt to authenticate using SASL (unknown error) 2019-11-14T20:37:08,818 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.java.util.http.client.NettyHttpClient.start()] on object[org.apache.druid.java.util.http.client.NettyHttpClient@11eec06b]. 2019-11-14T20:37:08,819 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.client.AbstractCuratorServerInventoryView.start() throws java.lang.Exception] on object[org.apache.druid.client.BatchServerInventoryView@19ad75e5]. 2019-11-14T20:37:08,820 INFO [main-SendThread(druid-zookeeper:2181)] org.apache.zookeeper.ClientCnxn - Socket connection established to druid-zookeeper/172.21.45.187:2181, initiating session 2019-11-14T20:37:08,837 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.client.BrokerServerView.start() throws java.lang.InterruptedException] on object[org.apache.druid.client.BrokerServerView@53982523]. 2019-11-14T20:37:08,841 INFO [main] org.apache.druid.client.BrokerServerView - BrokerServerView waiting for initialization. 2019-11-14T20:37:08,843 INFO [main-SendThread(druid-zookeeper:2181)] org.apache.zookeeper.ClientCnxn - Session establishment complete on server druid-zookeeper/172.21.45.187:2181, sessionid = 0x1000864ed690002, negotiated timeout = 30000 2019-11-14T20:37:08,860 INFO [main-EventThread] org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED 2019-11-14T20:37:08,925 INFO [ServerInventoryView-0] org.apache.druid.client.BatchServerInventoryView - Inventory Initialized 2019-11-14T20:37:08,929 INFO [main] org.apache.druid.client.BrokerServerView - BrokerServerView initialized in [88] ms. 2019-11-14T20:37:08,930 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.sql.calcite.schema.DruidSchema.start() throws java.lang.InterruptedException] on object[org.apache.druid.sql.calcite.schema.DruidSchema@150fc7a7]. 2019-11-14T20:37:08,941 INFO [main] org.apache.druid.sql.calcite.schema.DruidSchema - DruidSchema waiting for initialization. 2019-11-14T20:37:08,946 INFO [main] org.apache.druid.sql.calcite.schema.DruidSchema - DruidSchema initialized in [3] ms. 2019-11-14T20:37:08,946 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.java.util.http.client.NettyHttpClient.start()] on object[org.apache.druid.java.util.http.client.NettyHttpClient@77c66a4f]. 2019-11-14T20:37:08,946 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider.start()] on object[org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider@649b5891]. 2019-11-14T20:37:08,946 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - starting 2019-11-14T20:37:08,947 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - started 2019-11-14T20:37:08,947 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.curator.discovery.ServerDiscoverySelector.start() throws java.lang.Exception] on object[org.apache.druid.curator.discovery.ServerDiscoverySelector@5fb5ad40]. 2019-11-14T20:37:09,078 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.discovery.DruidLeaderClient.start()] on object[org.apache.druid.discovery.DruidLeaderClient@6afbe6a1]. 2019-11-14T20:37:09,079 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [COORDINATOR]. 2019-11-14T20:37:09,082 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [COORDINATOR]. 2019-11-14T20:37:09,082 INFO [main] org.apache.druid.discovery.DruidLeaderClient - Started. 2019-11-14T20:37:09,082 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.sql.calcite.schema.MetadataSegmentView.start()] on object[org.apache.druid.sql.calcite.schema.MetadataSegmentView@105db94d]. 2019-11-14T20:37:09,082 INFO [main] org.apache.druid.sql.calcite.schema.MetadataSegmentView - MetadataSegmentView Started. 2019-11-14T20:37:09,082 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.client.AbstractCuratorServerInventoryView.start() throws java.lang.Exception] on object[org.apache.druid.client.BatchServerInventoryView@1d99ee1b]. 2019-11-14T20:37:09,089 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.curator.discovery.ServerDiscoverySelector.start() throws java.lang.Exception] on object[org.apache.druid.curator.discovery.ServerDiscoverySelector@388be5fd]. 2019-11-14T20:37:09,102 INFO [ServerInventoryView-0] org.apache.druid.client.BatchServerInventoryView - Inventory Initialized 2019-11-14T20:37:09,106 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.discovery.DruidLeaderClient.start()] on object[org.apache.druid.discovery.DruidLeaderClient@6c8efde4]. 2019-11-14T20:37:09,106 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [OVERLORD]. 2019-11-14T20:37:09,108 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [OVERLORD]. 2019-11-14T20:37:09,154 INFO [main] org.apache.druid.discovery.DruidLeaderClient - Started. 2019-11-14T20:37:09,155 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.query.lookup.LookupReferencesManager.start()] on object[org.apache.druid.query.lookup.LookupReferencesManager@4b4814d0]. 2019-11-14T20:37:09,156 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - LookupExtractorFactoryContainerProvider is starting. 2019-11-14T20:37:09,156 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - No lookups to be loaded at this point 2019-11-14T20:37:09,166 INFO [main] org.apache.druid.query.lookup.LookupReferencesManager - LookupExtractorFactoryContainerProvider is started. 2019-11-14T20:37:09,168 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.server.listener.announcer.ListenerResourceAnnouncer.start()] on object[org.apache.druid.query.lookup.LookupResourceListenerAnnouncer@20216016]. 2019-11-14T20:37:09,170 INFO [main] org.apache.druid.server.listener.announcer.ListenerResourceAnnouncer - Announcing start time on [/druid/listeners/lookups/__default/http:172.30.33.175:8082] 2019-11-14T20:37:09,172 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [SERVER] 2019-11-14T20:37:09,172 INFO [main] org.apache.druid.server.initialization.jetty.JettyServerModule - Starting Jetty Server... 2019-11-14T20:37:09,182 INFO [NodeTypeWatcher[COORDINATOR]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Node[172.30.33.179:8081:DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/coordinator', host='172.30.33.179', bindOnHost=false, port=-1, plaintextPort=8081, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='COORDINATOR', services={}}] appeared. 2019-11-14T20:37:09,182 INFO [NodeTypeWatcher[COORDINATOR]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T20:37:09,182 INFO [main] org.eclipse.jetty.server.Server - jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 2019-11-14T20:37:09,185 INFO [NodeTypeWatcher[OVERLORD]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Node[172.30.33.179:8081:DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/coordinator', host='172.30.33.179', bindOnHost=false, port=-1, plaintextPort=8081, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='OVERLORD', services={}}] appeared. 2019-11-14T20:37:09,185 INFO [NodeTypeWatcher[OVERLORD]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T20:37:09,286 INFO [main] org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0 2019-11-14T20:37:09,286 INFO [main] org.eclipse.jetty.server.session - No SessionScavenger set, using defaults 2019-11-14T20:37:09,287 INFO [main] org.eclipse.jetty.server.session - node0 Scavenging every 600000ms 2019-11-14T20:37:09,395 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider as a provider class 2019-11-14T20:37:09,395 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider as a provider class 2019-11-14T20:37:09,396 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering org.apache.druid.server.BrokerQueryResource as a root resource class 2019-11-14T20:37:09,397 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering org.apache.druid.server.initialization.jetty.CustomExceptionMapper as a provider class 2019-11-14T20:37:09,397 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering org.apache.druid.server.initialization.jetty.ForbiddenExceptionMapper as a provider class 2019-11-14T20:37:09,397 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering org.apache.druid.server.initialization.jetty.BadRequestExceptionMapper as a provider class 2019-11-14T20:37:09,397 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Registering org.apache.druid.server.StatusResource as a root resource class 2019-11-14T20:37:09,400 INFO [main] com.sun.jersey.server.impl.application.WebApplicationImpl - Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM' 2019-11-14T20:37:09,502 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.initialization.jetty.CustomExceptionMapper to GuiceManagedComponentProvider with the scope "Singleton" 2019-11-14T20:37:09,504 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.initialization.jetty.ForbiddenExceptionMapper to GuiceManagedComponentProvider with the scope "Singleton" 2019-11-14T20:37:09,505 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.initialization.jetty.BadRequestExceptionMapper to GuiceManagedComponentProvider with the scope "Singleton" 2019-11-14T20:37:09,506 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding com.fasterxml.jackson.jaxrs.smile.JacksonSmileProvider to GuiceManagedComponentProvider with the scope "Singleton" 2019-11-14T20:37:09,516 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" 2019-11-14T20:37:09,907 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.http.security.StateResourceFilter to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,930 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.BrokerQueryResource to GuiceManagedComponentProvider with the scope "Undefined" 2019-11-14T20:37:09,935 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.http.BrokerResource to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,941 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.http.security.DatasourceResourceFilter to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,946 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.ClientInfoResource to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,948 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.client.HttpServerInventoryViewResource to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,950 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.http.security.ConfigResourceFilter to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,954 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.query.lookup.LookupListeningResource to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,957 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.query.lookup.LookupIntrospectionResource to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,959 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.sql.http.SqlResource to GuiceInstantiatedComponentProvider 2019-11-14T20:37:09,960 INFO [main] com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory - Binding org.apache.druid.server.StatusResource to GuiceManagedComponentProvider with the scope "Undefined" 2019-11-14T20:37:09,991 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@7e2c6fd6{/,null,AVAILABLE} 2019-11-14T20:37:10,005 INFO [main] org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@66ab924{HTTP/1.1,[http/1.1]}{0.0.0.0:8082} 2019-11-14T20:37:10,005 INFO [main] org.eclipse.jetty.server.Server - Started @11451ms 2019-11-14T20:37:10,005 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Starting lifecycle [module] stage [ANNOUNCEMENTS] 2019-11-14T20:37:10,006 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void org.apache.druid.curator.announcement.Announcer.start()] on object[org.apache.druid.curator.announcement.Announcer@6df2af02]. 2019-11-14T20:37:10,006 INFO [main] org.apache.druid.curator.announcement.Announcer - Starting announcer 2019-11-14T20:37:10,052 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeAnnouncer - Announcing [DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/broker', host='172.30.33.175', bindOnHost=false, port=-1, plaintextPort=8082, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='BROKER', services={lookupNodeService=LookupNodeService{lookupTier='default'}}}]. 2019-11-14T20:37:10,084 INFO [main] org.apache.druid.curator.discovery.CuratorDruidNodeAnnouncer - Announced [DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/broker', host='172.30.33.175', bindOnHost=false, port=-1, plaintextPort=8082, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='BROKER', services={lookupNodeService=LookupNodeService{lookupTier='default'}}}]. 2019-11-14T20:37:10,087 INFO [main] org.apache.druid.curator.discovery.CuratorServiceAnnouncer - Announcing service[DruidNode{serviceName='druid/broker', host='172.30.33.175', bindOnHost=false, port=-1, plaintextPort=8082, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}] 2019-11-14T20:37:10,132 WARN [main] org.apache.curator.utils.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead. 2019-11-14T20:37:10,136 INFO [main] org.apache.druid.java.util.common.lifecycle.Lifecycle - Successfully started lifecycle [module] 2019-11-14T20:38:43,330 INFO [ServerInventoryView-0] org.apache.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/172.30.33.136:8083 2019-11-14T20:38:43,331 INFO [ServerInventoryView-0] org.apache.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='172.30.33.136:8083', hostAndPort='172.30.33.136:8083', hostAndTlsPort='null', maxSize=300000000000, tier='_default_tier', type=historical, priority=0}] 2019-11-14T20:38:43,332 INFO [ServerInventoryView-0] org.apache.druid.curator.inventory.CuratorInventoryManager - Created new InventoryCacheListener for /druid/segments/172.30.33.136:8083 2019-11-14T20:38:43,332 INFO [ServerInventoryView-0] org.apache.druid.client.BatchServerInventoryView - New Server[DruidServerMetadata{name='172.30.33.136:8083', hostAndPort='172.30.33.136:8083', hostAndTlsPort='null', maxSize=300000000000, tier='_default_tier', type=historical, priority=0}] 2019-11-14T20:40:11,462 INFO [DruidSchema-Cache-0] org.apache.druid.sql.calcite.schema.DruidSchema - Refreshed metadata for dataSource[wikipedia] in 646 ms (1 segments queried, 0 segments left). 2019-11-14T21:12:00,999 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [HISTORICAL]. 2019-11-14T21:12:01,003 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [HISTORICAL]. 2019-11-14T21:12:01,058 INFO [NodeTypeWatcher[HISTORICAL]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Node[172.30.33.136:8083:DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/historical', host='172.30.33.136', bindOnHost=false, port=-1, plaintextPort=8083, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='HISTORICAL', services={dataNodeService=DataNodeService{tier='_default_tier', maxSize=300000000000, type=historical, priority=0}, lookupNodeService=LookupNodeService{lookupTier='default'}}}] appeared. 2019-11-14T21:12:01,058 INFO [NodeTypeWatcher[HISTORICAL]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T21:12:01,059 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [BROKER]. 2019-11-14T21:12:01,062 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [BROKER]. 2019-11-14T21:12:01,088 INFO [NodeTypeWatcher[BROKER]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Node[172.30.33.175:8082:DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/broker', host='172.30.33.175', bindOnHost=false, port=-1, plaintextPort=8082, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='BROKER', services={lookupNodeService=LookupNodeService{lookupTier='default'}}}] appeared. 2019-11-14T21:12:01,088 INFO [NodeTypeWatcher[BROKER]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T21:12:01,088 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [PEON]. 2019-11-14T21:12:01,090 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [PEON]. 2019-11-14T21:12:01,112 INFO [NodeTypeWatcher[PEON]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T21:12:01,112 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [ROUTER]. 2019-11-14T21:12:01,113 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [ROUTER]. 2019-11-14T21:12:01,122 INFO [NodeTypeWatcher[ROUTER]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Node[172.30.33.175:8888:DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/router', host='172.30.33.175', bindOnHost=false, port=-1, plaintextPort=8888, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='ROUTER', services={}}] appeared. 2019-11-14T21:12:01,123 INFO [NodeTypeWatcher[ROUTER]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T21:12:01,123 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [MIDDLE_MANAGER]. 2019-11-14T21:12:01,124 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [MIDDLE_MANAGER]. 2019-11-14T21:12:01,137 INFO [NodeTypeWatcher[MIDDLE_MANAGER]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Node[172.30.33.136:8091:DiscoveryDruidNode{druidNode=DruidNode{serviceName='druid/middleManager', host='172.30.33.136', bindOnHost=false, port=-1, plaintextPort=8091, enablePlaintextPort=true, tlsPort=-1, enableTlsPort=false}, nodeType='MIDDLE_MANAGER', services={workerNodeService=WorkerNodeService{ip='druid-data-0.druid.observability.svc.cluster.local', capacity=4, version='0'}}}] appeared. 2019-11-14T21:12:01,137 INFO [NodeTypeWatcher[MIDDLE_MANAGER]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T21:12:01,137 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Creating NodeTypeWatcher for nodeType [INDEXER]. 2019-11-14T21:12:01,138 INFO [sql[dfab96d5-face-4acd-8553-de672dd6c106]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider - Created NodeTypeWatcher for nodeType [INDEXER]. 2019-11-14T21:12:01,145 INFO [NodeTypeWatcher[INDEXER]] org.apache.druid.curator.discovery.CuratorDruidNodeDiscoveryProvider$NodeTypeWatcher - Received INITIALIZED in node watcher. 2019-11-14T21:13:14,900 INFO [qtp1093707336-109] org.apache.druid.guice.JsonConfigurator - Loaded class[class org.apache.druid.client.DruidServerConfig] from props[druid.server.] as [org.apache.druid.client.DruidServerConfig@17cc5661]

ceastman-ibm commented 4 years ago

so i checked my druid cluster this morning and now all the servers are showing up:

Screen Shot 2019-11-18 at 8 05 12 AM
surekhasaharan commented 4 years ago

Nice, thanks for updating!

ceastman-ibm commented 4 years ago

@surekhasaharan looks like its an issue with zookeeper running on master, when i switched to using an external zookeeper service everything is working as intended.