etcd-io/discovery.etcd.io

Public metrics

Public metrics are hosted at: https://grafana.prod.discovery.etcd.io/d/uiLwPyPWk/discoveryserver?orgId=2

discovery.etcd.io Kubernetes Configurations

This repo contains the code to provision the infrastructure and the Kubernetes configurations to operate the public discovery.etcd.io service.

Requirements

Helm ~> v3.0.0 - Install latest version for your OS.
Terraform ~> v0.12.15 Please download the proper package for your operating system and architecture.

Building the infrastructure

The infrastructure is built using cloudkite terraform modules, which are used to provision infrastructure in Google Cloud Platform. The following modules have been used:

vpc: The vpc module contains Terraform code to provision a GCP Virtual Private Cloud. See VPC docs.
gke: The folder contains Terraform code to deploy a GKE Private Cluster.

Provisioning a VPC and deploying a GKE cluster per environment

The infrastructure main code is created per environment, and there are two environments:

dev
prod

Choose an environment - that is, move to the dev or to prod folder in order to run Terraform commands.

Next step is to apply Terraform for the chosen environment. To ensure that it is configured correctly, apply it and get the expected output, go to the project's terraform folder and follow the README instructions.

Manual Step

Once dev and prod infrastructure is built, it is required to update IAM policies of artifacts.<dev-project>.appspot.com GCS bucket adding both dev and prod gke_service_accounts as members with the role roles/storage.objectViewer. Only after it, clusters in both environments will be able to pull images from gcr.

Gsutil command to update the IAM policy:

gsutil iam ch serviceAccount:[SERVICE_ACCOUNT_EMAIL]:roles/storage.objectViewer gs://artifacts.<dev-project>.appspot.com

Note: if you get an error that the artifacts.<dev-project>.appspot.com does not exit, push an image and it will be created.

After applying terraform, a GKE cluster will be up and running in the VPC created. Now the cluster is ready to get deployments.

Install Releases with Helm

To get the public discovery service running, the following releases have to be installed:

Nginx Ingress Controller: Used for routing traffic from beyond the cluster to internal Kubernetes Services. To install follow instructions in README.
certmanager: Is the TLS/SSL certificate management controller, and to get it deployed follow the README.
etcd-operator: Is used to configure and manage etcd clusters. This is a pre-requisite to have configured properly the discoveryserver release. To install it follow instructions in README.
discoveryserver: Is a service that bootstrap new etcd clusters using an existing one. This service helps when the IPs of your cluster peers are not known ahead of time. To install the release follow instructions in README.

block diagram of architecture

Debugging

Hit the discovery service via kubectl proxy

kubectl proxy
curl http://localhost:8001/api/v1/namespaces/default/services/discoveryserver/proxy/new

Execute etcdctl on the cluster

kubectl exec -it $(kubectl get pods -l app=etcd -o jsonpath='{.items[0].metadata.name}')  -- /usr/local/bin/etcdctl watch '' --prefix