Closed gregorybleiker closed 3 years ago
AdheipSingh
commented
3 years ago do you see any error logs on the operator pod ? kubectl describe druid ? operator events ? kubectl get druid ? do you the CR object ?
Once you apply a druid CR, a cluster is should be up.
gregorybleiker
commented
3 years ago Here are the logs of the operator pod:
2021-09-01T07:14:24.692Z INFO controller-runtime.metrics metrics server is starting to listen {"addr": ":8080"}
2021-09-01T07:14:24.693Z INFO setup starting manager
2021-09-01T07:14:24.694Z INFO controller-runtime.manager starting metrics server {"path": "/metrics"}
2021-09-01T07:14:24.694Z INFO controller-runtime.manager.controller.druid Starting EventSource {"reconciler group": "druid.apache.org", "reconciler kind": "Druid", "source": "kind source: /, Kind="}
2021-09-01T07:14:24.795Z INFO controller-runtime.manager.controller.druid Starting Controller {"reconciler group": "druid.apache.org", "reconciler kind": "Druid"}
2021-09-01T07:14:24.795Z INFO controller-runtime.manager.controller.druid Starting workers {"reconciler group": "druid.apache.org", "reconciler kind": "Druid", "worker count": 1}
2021-09-01T07:15:03.007Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T07:24:07.934Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T07:39:41.901Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T07:51:42.414Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T07:57:47.551Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T08:11:33.007Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T08:23:39.399Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T08:32:58.536Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T08:47:45.953Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T09:00:32.289Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T09:10:25.688Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T09:24:10.752Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concile
2021-09-01T09:39:37.001Z INFO druid_operator_handler druid operator will not re-concile namespace [default], alter DENY_LIST to re-concilekubectl describe druid:
Name: tiny-cluster
Namespace: default
Labels: <none>
Annotations: <none>
API Version: druid.apache.org/v1alpha1
Kind: Druid
Metadata:
Creation Timestamp: 2021-09-01T07:15:02Z
Generation: 1
Managed Fields:
API Version: druid.apache.org/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:common.runtime.properties:
f:commonConfigMountPath:
f:env:
f:image:
f:jvm.options:
f:log4j.config:
f:nodes:
.:
f:brokers:
.:
f:druid.port:
f:extra.jvm.options:
f:nodeConfigMountPath:
f:nodeType:
f:replicas:
f:runtime.properties:
f:coordinators:
.:
f:druid.port:
f:extra.jvm.options:
f:nodeConfigMountPath:
f:nodeType:
f:replicas:
f:runtime.properties:
f:historicals:
.:
f:druid.port:
f:extra.jvm.options:
f:nodeConfigMountPath:
f:nodeType:
f:replicas:
f:runtime.properties:
f:routers:
.:
f:druid.port:
f:extra.jvm.options:
f:nodeConfigMountPath:
f:nodeType:
f:replicas:
f:runtime.properties:
f:podAnnotations:
.:
f:dummykey:
f:podLabels:
.:
f:environment:
f:release:
f:readinessProbe:
.:
f:httpGet:
.:
f:path:
f:port:
f:securityContext:
.:
f:fsGroup:
f:runAsGroup:
f:runAsUser:
f:services:
f:startScript:
f:volumeMounts:
f:volumes:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-09-01T07:15:02Z
Resource Version: 713
UID: da36bb91-df1a-445a-b644-89c23467e515
Spec:
common.runtime.properties:
# Zookeeper
druid.zk.service.host=tiny-cluster-zk-0.tiny-cluster-zk
druid.zk.paths.base=/druid
druid.zk.service.compress=false
# Metadata Store
druid.metadata.storage.type=derby
druid.metadata.storage.connector.connectURI=jdbc:derby://localhost:1527/druid/data/derbydb/metadata.db;create=true
druid.metadata.storage.connector.host=localhost
druid.metadata.storage.connector.port=1527
druid.metadata.storage.connector.createTables=true
# Deep Storage
druid.storage.type=local
druid.storage.storageDirectory=/druid/deepstorage
#
# Extensions
#
druid.extensions.loadList=["druid-kafka-indexing-service"]
#
# Service discovery
#
druid.selectors.indexing.serviceName=druid/overlord
druid.selectors.coordinator.serviceName=druid/coordinator
druid.indexer.logs.type=file
druid.indexer.logs.directory=/druid/data/indexing-logs
druid.lookup.enableLookupSyncOnStartup=false
Common Config Mount Path: /opt/druid/conf/druid/cluster/_common
Env:
Name: POD_NAME
Value From:
Field Ref:
Field Path: metadata.name
Name: POD_NAMESPACE
Value From:
Field Ref:
Field Path: metadata.namespace
Image: apache/druid:0.20.0
jvm.options: -server
-XX:MaxDirectMemorySize=10240g
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Dlog4j.debug
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
-Djava.io.tmpdir=/druid/data
log4j.config: <?xml version="1.0" encoding="UTF-8" ?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{ISO8601} %p [%t] %c - %m%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Nodes:
Brokers:
druid.port: 8088
extra.jvm.options: -Xmx512M
-Xms512M
Node Config Mount Path: /opt/druid/conf/druid/cluster/query/broker
Node Type: broker
Replicas: 1
runtime.properties: druid.service=druid/broker
# HTTP server threads
druid.broker.http.numConnections=5
druid.server.http.numThreads=10
# Processing threads and buffers
druid.processing.buffer.sizeBytes=1
druid.processing.numMergeBuffers=1
druid.processing.numThreads=1
druid.sql.enable=true
Coordinators:
druid.port: 8088
extra.jvm.options: -Xmx512M
-Xms512M
Node Config Mount Path: /opt/druid/conf/druid/cluster/master/coordinator-overlord
Node Type: coordinator
Replicas: 1
runtime.properties: druid.service=druid/coordinator
# HTTP server threads
druid.coordinator.startDelay=PT30S
druid.coordinator.period=PT30S
# Configure this coordinator to also run as Overlord
druid.coordinator.asOverlord.enabled=true
druid.coordinator.asOverlord.overlordService=druid/overlord
druid.indexer.queue.startDelay=PT30S
druid.indexer.runner.type=local
Historicals:
druid.port: 8088
extra.jvm.options: -Xmx512M
-Xms512M
Node Config Mount Path: /opt/druid/conf/druid/cluster/data/historical
Node Type: historical
Replicas: 1
runtime.properties: druid.service=druid/historical
druid.server.http.numThreads=5
druid.processing.buffer.sizeBytes=536870912
druid.processing.numMergeBuffers=1
druid.processing.numThreads=1
# Segment storage
druid.segmentCache.locations=[{\"path\":\"/druid/data/segments\",\"maxSize\":10737418240}]
druid.server.maxSize=10737418240
Routers:
druid.port: 8088
extra.jvm.options: -Xmx512M
-Xms512M
Node Config Mount Path: /opt/druid/conf/druid/cluster/query/router
Node Type: router
Replicas: 1
runtime.properties: druid.service=druid/router
# HTTP proxy
druid.router.http.numConnections=10
druid.router.http.readTimeout=PT5M
druid.router.http.numMaxThreads=10
druid.server.http.numThreads=10
# Service discovery
druid.router.defaultBrokerServiceName=druid/broker
druid.router.coordinatorServiceName=druid/coordinator
# Management proxy to coordinator / overlord: required for unified web console.
druid.router.managementProxy.enabled=true
Pod Annotations:
Dummykey: dummyval
Pod Labels:
Environment: stage
Release: alpha
Readiness Probe:
Http Get:
Path: /status/health
Port: 8088
Security Context:
Fs Group: 1000
Run As Group: 1000
Run As User: 1000
Services:
Spec:
Cluster IP: None
Type: ClusterIP
Start Script: /druid.sh
Volume Mounts:
Mount Path: /druid/data
Name: data-volume
Mount Path: /druid/deepstorage
Name: deepstorage-volume
Volumes:
Empty Dir:
Name: data-volume
Host Path:
Path: /tmp/druid/deepstorage
Type: DirectoryOrCreate
Name: deepstorage-volume
Events: <none>kubectl get druid:
NAME AGE
tiny-cluster 150mThe CR seems to be there:
Not quite sure how to get operator events, sorry
AdheipSingh
commented
3 years ago I see, so by default the operator does not reconcile default and kube-system namespace, and it seems you have deployed your tiny cluster yaml in default ns. So either you can edit your deployment file and remove default from deny list env, or deploy your CR in another ns. It should work.
gregorybleiker
commented
3 years ago Ahhh!! Thank you @AdheipSingh ! So the only problem was that I had to use
kubectl apply -f examples/tiny-cluster.yaml -n druidinstead of
kubectl apply -f examples/tiny-cluster.yamlNow it is running. 👍
I'm using Docker Desktop on Windows with Kubernetes enabled and also WSL2 enabled.
I have tried to setup the tiny-cluster but I am stuck. It seems the operator is installed and also the Zookeeper instance, but no druid related resources get created. I installed the operator through the helm chart. At what time should the components (like the router etc) be created? Also, does it matter in what namespace the tiny-cluster gets deployed?
Thanks for any help!