This repo has the scripts to provision and configure a Kubernetes cloud infrastructure and install Keptn, the open-source framework for event-based, automated continuous operations, as well as onboarding a micro service based order processing demo application into Keptn. The other repos in this GitHub organization contain the demo application source code. The purpose of this effort is for demonstrations of the Keptn platfor on various Kubernetes platforms.
Currently, Keptn 0.4.0 and these keptn-orders-setup scripts support only Google GKE and Azure AKS.
Branches within this repo are keep in sync with Keptn releases. Master branch is the latest fully regression tested. This branch tested for Keptn 0.4.0
This Demo is maintained by rob.jahn@dynatrace.com. If you have questions or feedback, you can reach out to me.
Watch a video demo of the install and onboarding of Keptn Orders demo app on Amazon AKS - part1 and Part two
Much of this same information is in this readme, but here are two blogs that give an overivew the setup.
There are 4 components, a front-end and 3 backend services. The front-end look like this.
Once monitored by Dynatrace, a multi-tier call flow will be available such as shown below.
Other details:
Table of Contents generated by this utility https://github.com/ekalinin/github-markdown-toc
docker run -it vemonet/markdown-toc-generator https://github.com/keptn-orders/keptn-orders-setup/blob/master/README.md
repo
scope added. You can also reference the keptn docsKeptn expects all the code repos and project files to be in the same GitHub Organization. So create a github new github organization for the keptn-orders for Keptn to use and for the keptn-orders application repos to be forked. See details in the github docs
Suggested gihub organization name: <your last name>-keptn-orders-<cloud provider>
for examaple jahn-keptn-orders-aks
NOTE: If the 'orders-project' repo already exists in your personal github org, there may be errors when you onboard again. So delete the repo if it exists.
The following set of tools are required by the installation scripts and interacting with the environment. The setup scripts will install these automatically on the bastion host, but below is a listing for reference.
All platforms
Google additional tools
Azure additional tools
Amazon additional tools
See these instructions for provisioning an ubuntu 16.04 LTS host on the targeted cloud provider. The setup scripts assume this version, so don't adjust this.
There are multiple scripts used for the setup and they must be run the right order. Just run the setup script that will prompt you with menu choices.
./setup.sh <deployment type>
NOTE: Valid 'deployment type' argument values are:
The setup menu should look like this:
====================================================
SETUP MENU
====================================================
1) Install Prerequisites Tools
2) Enter Installation Script Inputs
3) Provision Kubernetes cluster
4) Install Keptn
5) Install Dynatrace
6) Fork keptn-orders Repos
7) Onboard keptn-orders App to Keptn
8) Setup HA Proxy to Keptn Bridge
----------------------------------------------------
10) Validate Kubectl
11) Validate Prerequisite Tools
----------------------------------------------------
20) Show Orders App
21) Show Keptn
22) Show Dynatrace
----------------------------------------------------
30) Send Keptn Artifact Events
----------------------------------------------------
99) Delete Kubernetes cluster
====================================================
Please enter your choice or <q> or <return> to exit
NOTE: each script will log the console output into the logs/
subfolder.
This will install the required unix tools such as kubectl, jq, cloud provider CLI.
At the end if the installation, the script will
NOTE: You can re-run both 'Install Prerequisites Tools' or 'Validate Prerequisite Tools' anytime as required.
Before you do this step, be prepared with your github credentials, dynatrace tokens, and cloud provider project information available.
This will prompt you for values that are referenced in the remaining setup scripts. Inputted values are stored in creds.json
file. For example on GKE the menus looks like:
===================================================================
Please enter the values for provider type: Google GKE:
===================================================================
Dynatrace Host Name (e.g. abc12345.live.dynatrace.com)
(current: DYNATRACE_HOSTNAME_PLACEHOLDER) :
Dynatrace API Token (current: DYNATRACE_API_TOKEN_PLACEHOLDER) :
Dynatrace PaaS Token (current: DYNATRACE_PAAS_TOKEN_PLACEHOLDER) :
GitHub User Name (current: GITHUB_USER_NAME_PLACEHOLDER) :
GitHub Personal Access Token (current: PERSONAL_ACCESS_TOKEN_PLACEHOLDER) :
GitHub User Email (current: GITHUB_USER_EMAIL_PLACEHOLDER) :
GitHub Organization (current: GITHUB_ORG_PLACEHOLDER) :
Google Project (current: GKE_PROJECT_PLACEHOLDER) :
Cluster Name (current: CLUSTER_NAME_PLACEHOLDER) :
Cluster Zone (eg.us-east1-b) (current: CLUSTER_ZONE_PLACEHOLDER) :
Cluster Region (eg.us-east1) (current: CLUSTER_REGION_PLACEHOLDER) :
This will provision a Cluster on the specified cloud deployment type using the platforms CLI. This script will take several minutes to run and you can verify the cluster was created with the the cloud provider console.
The cluster will take 5-10 minutes to provision.
This script at the end will run the 'Validate Kubectl' script.
This will install the Keptn control plane components into your cluster. The install will take 5-10 minutes to perform.
NOTE: Internally, this script will perform the following:
2-enterInstallationScriptInputs.sh
script to create the creds.json filekeptn install -c=creds.json --platform=<Cluster>
This will install the Dynatrace OneAgent Operator into your cluster. The install will take 3-5 minutes to perform.
NOTE: Internally, this script will perform the following:
2-enterInstallationScriptInputs.sh
script to create the creds_dt.json file/deploy/scripts/deployDynatraceOn<Platform>.sh
script in the dynatrace-service folderThis will fork the keptn-orders application into the github organization you specified when you called 'Enter Installation Script Inputs' step.
Internally, this script will:
hub
unix git utility to fork each repositoriesThis script will onboard the orders application using the keptn
CLI tool and the keptn onboarding files found in the keptn-onboarding/
folder.
Internally, this script will:
You can verify the onbaording was complete by reviewing the 'orders-project' within your personal git org.
The keptn’s bridge provides an easy way to browse all events that are sent within keptn and to filter on a specific keptn context. When you access the keptn’s bridge, all keptn entry points will be listed in the left column. Please note that this list only represents the start of a deployment of a new artifact and, thus, more information on the executed steps can be revealed when you click on one event.
The keptn’s bridge is not publicly accessible, but can be retrieved using kubernetes port-forwarding using the kubectl port-forward
command. This script will install haproxy service on the bastion host, configure it with basic authentication, listen on port 80 and forward it to kubernetes listening on port 9000.
The script will output the kubectl port-forward
command to run as well as the URL to open in a browser to view the Keptn Bridge. I recommend making a second terminal window since the command runs in a continuous loop.a
Sample output:
======================================================================
Creating new /etc/haproxy/haproxy.cfg
Restarting haproxy
======================================================================
Start Keptn Bridge with this command:
while true; do kubectl port-forward svc/bridge -n keptn 9000:8080; done
View bridge @ http://123.123.123.123/#/
Recommend using a seperate SSH terminal session since the kubectl port-forward
remains running.
This script will attempt to 'get pods' using kubectl.
This script will look for the existence of required prerequisite tools. It does NOT check for version just the existence of the script.
These scripts are helpful when using and reviewing status of your environment. Just run the helper script that will prompt you with menu choices.
Displays the deployed orders application pods and urls to access the application
Displays the Keptn pods and ingress gateway
Displays the Dynatrace pods
Keptn deployments start with a "new-artifact" event. This option will prompt you for which servier and version to send an artifact for. The script will call the keptn CLI send event command. This is an example of deploying version 1 for the front-end service.
===================================================================
Please enter the image version to send, example: 1
Images with skip will not send an event
===================================================================
frontend (default:skip) : 1
order service (default:skip) :
catalog service (default:skip) :
customer service (default:skip) :
Use the 'Show App' helper script to get the pod status and the URLs to the application.
Fastest way to remove everything is to delete your cluster using this script. Be careful when you run this as to not lose your work.
The keptn-orders application has pre-built problems programmed within different versions. See source in the keptn-orders repo. Each version for each service, has pre-built docker images that are published to docker hub.
This is a summary of the versions followed by a description of the problem scenarios.
Service | Branch/Docker Tag | Description |
---|---|---|
front-end | 1 | Normal behavior |
catalog-service | 1 | Normal behavior |
customer-service | 1 | Normal behavior |
order-service | 1 | Normal behavior |
customer-service | 2 | High Response time for /customer/list.html |
order-service | 2 | 50% exception for /order/line URL and n+1 back-end calls for /order/form.html |
customer-service | 3 | Normal behavior |
order-service | 3 | Normal behavior |
# use the keptn cli to deploy new image
keptn send event new-artifact --project=orders-project --service=customer-service --image=robjahn/keptn-orders-customer-service --tag=2
# use the keptn cli to deploy new image
keptn send event new-artifact --project=orders-project --service=order-service --image=robjahn/keptn-orders-order-service --tag=2