A dashboard to monitor the GOV.UK PaaS platform.
Uses Steampipe to access all the component parts from the underlying infrastructure in AWS and Aiven through to the Cloud Foundry core of the paas, its code base and supporting tools.
see kanban board for current plan.
Originally a :fire: firebreak experiment.
Steampipe provides a SQL layer on top of a wide range of cloud platform services that have apis using a postgresql foreign data wrapper.
This dashboard uses steampipe.io to build a set of dashboards over GOV.UK PaaS and its infrastructure.
It uses the Cloud Foundry CLI to access the Cloud Foundry API and list resources such as orgs, apps,services, the data is saved locally as csv files and accessed from a local steampipe dashboard running at http://localhost:9194
It uses the plugins configured in dashboards/mod.sp
The dashboards pull data from the underlying csv files using postgres SQL and render the results as a dashboard.
cf login --sso
cf curl /v3/foobar | in2csv -f json -k resources
converting JSON into CSV using csvkit's in2csv
gds aws paas-prod-ro
steampipe query query.sql -- output csv
Render results locally accessing data from plugins, running SQL queries against the normalised data using postgresql
Read about the data model
Assumes you are on a mac with homebrew installed with /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# homebrew packages
brew install cf-cli@8 # Cloud Foundry CLI
brew install gawk # GNU awk
brew install gh # GitHub CLI
brew install glow # Glow CLI for markdown
brew install gnu-sed # GNU sed
brew install jq # JSON wrangling tool
brew install steampipe # make cloud apis queryble via SQL
brew install yq # YAML tools
# python tools
pip3 install csvkit # csv wrangling tools
pip3 install visidata # data wrangling swiss army penknife tool
that you are on the VPN
you have a GOV.UK PaaS account with global auditor permissions and you are logged in using make login
you are a gds-user with permisssions to assume role into all the paas infrastructure AWS accounts
you have the aiven CLI configured with an access token to list services
git clone https://github.com/alphagov/paas-steampipe-dashboard
cd paas-steampipe-dashboard
make dependencies
to install all the necessary packages
see config for examples
vim ~/.steampipe/config
you need creds to access the AWS accounts
gds aws paas-prod-ro -- make data
this logs into to AWS and Cloud Foundry, extracts data locally in csv format
make dashboard
to run the dashboard with the current data and launch dashboard at http://localhost:9194
does not need Cloud Foundry or AWS creds because the dashboard uses local data
use steampipe service
start to run locally and steampipe service status
to get creds to configure your favourite postgres tools.