Services are failing to show in the web console

[premal]

Hi, I followed the instructions on https://github.com/druid-io/druid-operator/blob/master/docs/getting_started.md to spin up the tiny cluster on EKS

This is the output of kubectl get all

NAME                                    READY   STATUS    RESTARTS   AGE
pod/druid-operator-85d6749dd5-w6ghc     1/1     Running   0          52m
pod/druid-tiny-cluster-brokers-0        1/1     Running   0          51m
pod/druid-tiny-cluster-coordinators-0   1/1     Running   0          51m
pod/druid-tiny-cluster-historicals-0    1/1     Running   0          51m
pod/druid-tiny-cluster-routers-0        1/1     Running   0          51m
pod/tiny-cluster-zk-0                   1/1     Running   0          51m

NAME                                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/druid-operator-metrics            ClusterIP   172.20.165.207   <none>        8383/TCP,8686/TCP            52m
service/druid-tiny-cluster-brokers        ClusterIP   None             <none>        8088/TCP                     51m
service/druid-tiny-cluster-coordinators   ClusterIP   None             <none>        8088/TCP                     51m
service/druid-tiny-cluster-historicals    ClusterIP   None             <none>        8088/TCP                     51m
service/druid-tiny-cluster-routers        ClusterIP   None             <none>        8888/TCP                     51m
service/kubernetes                        ClusterIP   172.20.0.1       <none>        443/TCP                      34h
service/tiny-cluster-zk                   ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   51m

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/druid-operator   1/1     1            1           52m

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/druid-operator-85d6749dd5   1         1         1       52m

NAME                                               READY   AGE
statefulset.apps/druid-tiny-cluster-brokers        1/1     51m
statefulset.apps/druid-tiny-cluster-coordinators   1/1     51m
statefulset.apps/druid-tiny-cluster-historicals    1/1     51m
statefulset.apps/druid-tiny-cluster-routers        1/1     51m
statefulset.apps/tiny-cluster-zk                   1/1     51m

I forwarded the port to load the web console

kubectl port-forward -n default service/druid-tiny-cluster-routers 8088:8888 

In the UI, the services do not show up

[himanshug]

It would be great to have example specs that create functioning Druid clusters. tiny-cluster spec probably isn't there yet and needs to evolve.

you can find some prod cluster references in https://github.com/druid-io/druid-operator/issues/12 , hth

[premal]

Hi @himanshug . Thanks for replying. I'm using the tiny-cluster spec as per these instructions - https://github.com/druid-io/druid-operator/blob/master/docs/getting_started.md#deploy-a-sample-druid-cluster

It comes from here - https://github.com/druid-io/druid-operator/blob/master/examples/tiny-cluster.yaml

[AdheipSingh]

@premal you can refer here https://gist.github.com/AdheipSingh/a87eb276badd2c614973d7aa56b07cbd , few folks have used this. You will need to change runtime/common properties according to your cluster requirements.

[cswarth]

What's the status of this issue? Are we meant to experiment with the cluster config in the above gist in place of "tiny-cluster.yaml"?

I see the same behavior described above, services in accessible.

Also wondering why Load Data button is disabled? I was hoping to walk through some of the tutorial to load data from S3 into druid running in EKS.

[himanshug]

@cswarth @premal you can try the updated tiny-cluster.yaml introduced in #120 , "Example Data" flow works ..For EKS and S3 related stuff you might have to update the configuration to load s3 extensions and change deep storage location etc. Hopefully this is a better starting point.

[cswarth]

Nice - your updated tiny-cluster.yaml in #120 worked for me on a single m5.xlarge EKS node. The dashboard is fully functional, services are visible, and SQL is enabled.

It has been very instructive to see the difference between the original tiny-cluster.yaml that required something like 5 nodes, your updated tiny-cluster.yaml that requires only one node, and the one in the gist above that looks like requires about 8 nodes, and only works on kops deployments due to a dependency on kops-ssd-1-17.

[ to clarify, the original had cpu resource requirements, and since I was running on relatively small instances, it had to run multiple nodes. Presumably if I had run it on a 32 cpu instance it could have all fit on one.]

fwiw, I was not able to get your updated tiny-cluster.yaml to work in minikube on macOS. The druid-tiny-cluster-historicals-0 pod never became ready and the last couple log lines were uninformative. Not worried about resolving, but just fyi.

[himanshug]

@cswarth thanks for verifying that .. not sure why it wouldn't work on minikube .. can you post the steps you did to run on minikube (minikube version, minikube start command etc etc)... I might try and repro it sometime.