search

hashicorp / demo-consul-101

Tutorial code and binaries for the HashiCorp Consul beginner course.
Mozilla Public License 2.0
81 stars 54 forks source link

Unable to connect to counting service #2

Open vikramkhatri opened 5 years ago

vikramkhatri commented 5 years ago

Env: IBM Cloud Private just running one node.

# kubectl version --short
Client Version: v1.12.4+icp
Server Version: v1.12.4+icp

Installed Consul 0.8.1 in Kubernetes. I had difficulty in installing as port 8500 and 8600 are in use if using ICP. But, after I modified the targetPort in service and daemonset, the pods came up and I can see the service in UI.

I followed your example step by step but I can't seem to connect with the counting service from the dashboard.

The tutorial did not mention anything about creating dns stub but readme says it.

So as per this link https://www.consul.io/docs/platform/k8s/dns.html I created this.

This was my coredns:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    addonmanager.kubernetes.io/mode: EnsureExists
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
      errors
      health
      kubernetes cluster.local in-addr.arpa ip6.arpa {
         pods insecure
         upstream
         fallthrough in-addr.arpa ip6.arpa
      }
      prometheus :9153
      proxy . /etc/resolv.conf
      cache 30
      reload
      loadbalance
    }
    consul {
      errors
      cache 30
      proxy . 10.0.0.111
    }

The consul dns is running on IP 10.0.0.111. I tried DNS test and I did not succeed for consul.service.consul and I do not know from where this name comes.

This is my setup for consul.

# k get all
NAME                                   READY   STATUS    RESTARTS   AGE
pod/consul-consul-bgt6d                1/1     Running   0          6h42m
pod/consul-consul-server-0             1/1     Running   0          6h42m
pod/consul-consul-server-1             1/1     Running   0          6h42m
pod/consul-consul-server-2             1/1     Running   0          6h42m
pod/counting                           1/1     Running   0          68m
pod/dashboard                          1/1     Running   0          68m
pod/emojify-api-c9dcb8c8c-xdsb8        2/2     Running   15         46m
pod/emojify-cache-594dbc9cb6-mrj7n     2/2     Running   0          46m
pod/emojify-facebox-6b995cb9bd-smz49   2/2     Running   0          46m
pod/emojify-ingress-79bf696c95-qxqxq   2/2     Running   0          46m
pod/emojify-website-6f45f44fd5-z7csg   2/2     Running   0          46m

NAME                                      TYPE           CLUSTER-IP   EXTERNAL-IP       PORT(S)                                                                   AGE
service/consul-consul-dns                 ClusterIP      10.0.0.111   <none>            53/TCP,53/UDP                                                             6h42m
service/consul-consul-server              ClusterIP      None         <none>            8500/TCP,8301/TCP,8301/UDP,8302/TCP,8302/UDP,8300/TCP,8611/TCP,8611/UDP   6h42m
service/consul-consul-ui                  ClusterIP      10.0.0.198   <none>            80/TCP                                                                    6h42m
service/dashboard-service-load-balancer   LoadBalancer   10.0.0.62    192.168.142.249   80:31574/TCP                                                              68m
service/emojify-ingress                   NodePort       10.0.0.26    <none>            80:30000/TCP                                                              46m

NAME                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/consul-consul   1         1         1       1            1           <none>          6h42m

NAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/emojify-api       1         1         1            1           46m
deployment.apps/emojify-cache     1         1         1            1           46m
deployment.apps/emojify-facebox   1         1         1            1           46m
deployment.apps/emojify-ingress   1         1         1            1           46m
deployment.apps/emojify-website   1         1         1            1           46m

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/emojify-api-c9dcb8c8c        1         1         1       46m
replicaset.apps/emojify-cache-594dbc9cb6     1         1         1       46m
replicaset.apps/emojify-facebox-6b995cb9bd   1         1         1       46m
replicaset.apps/emojify-ingress-79bf696c95   1         1         1       46m
replicaset.apps/emojify-website-6f45f44fd5   1         1         1       46m

NAME                                    DESIRED   CURRENT   AGE
statefulset.apps/consul-consul-server   3         3         6h42m

The counting service responds.

# k exec -it counting -- curl http://localhost:9001
{"count":1,"hostname":"counting"}

The logs from dashboard.

# k logs dashboard
Starting server on http://0.0.0.0:9002
(Pass as PORT environment variable)
Using counting service at http://localhost:9001
Starting websocket server...
New client connected
Fetched count -1
Fetched count -1
New client connected
Fetched count -1
Fetched count -1
New client connected

The UI shows -1 and it says Counting service is unreachable.

What did I miss? And how to test dns stub and if services are discoverable. I do not see counting and dashboard in the UI except consul.

Lord-Y commented 5 years ago

Same for me on a GKE cluster (GCloud). Really don't know what to do and I know no errors in stackdriver logging for my GKE cluster

Lord-Y commented 5 years ago

Alright we figure out the issue. Consul need talk with kubernetes masters so for this dummy test we created an open firewall rules and then ... the light come out :). Everything works, I got my second container etc etc. Hope that can help @vikramkhatri

Lord-Y commented 5 years ago

On GKE with shared vpc you need to open 8080 port between masters and pods.8080 port is used by consul webhook injector