This adds docker support for building images for ct-test-srv, ct-woodpecker, and ct-malformed, and an end-to-end environment for testing ct-woodpecker and Prometheus alerts backed by its metrics.
The provided docker-compose.yml can be used to create a ct-test-srv container providing two certificate transparency logs log-one and log-two. A ct-woodpecker container is configured to monitor both logs storing the data in the mounted sqlite3 db storage.sqlite.
A prometheus container is configured to scrape the ct-woodpecker container metrics, and to use an alertmanager container to process any alerts. For now the alertmanager container uses an /echo endpoint on one of the ct-test-srv logs as a webhook POST target so that alerts are printed to stdout via the ct-test-srv. The prometheus container comes configured with the alerts from https://github.com/letsencrypt/ct-woodpecker/pull/70 as a starting-point (thanks @pgporada!)
To allow testing alert rules the ct-test-srv now has a way to add generic mock responses for any path. This can be used (for e.g.) to configure one of the logs to return a 500 error for /ct/v1/add-chain for a period of time.
To use the docker environment, first make sure you have a storage.sqlite db created in the project root:
rm storage.sqlite 2>/dev/null ; cat storage/schema.sqlite | sqlite3 ./storage.sqlite
Then start the containers:
docker-compose up
You can access Prometheus at:
http://localhost:9090
You can break certificate submission for log-two by running:
curl -X POST -d '{"path":"/ct/v1/add-chain","code":404,"response":{"error":"oh noes!"}}' localhost:4601/add-mock
Shortly afterwards (2-4m) you can expect the CertSubmissionErrors alert to be firing in http://localhost:9090/alerts based on the ct-woodpecker container being unable to submit certificates to log-two.
You can cause the alert to recover by fixing log-two's certificate submission by running:
curl -X POST -d '{"path":"/ct/v1/add-chain"}' localhost:4601/clear-mock
Prometheus scrape data will be maintained in Docker volume called ct-woodpecker_prometheus_data. If you remove your docker-compose containers make sure to also docker volume rm ct-woodpecker_prometheus_data to reset the Prometheus data volume.
coveralls
commented
5 years ago
This adds docker support for building images for
ct-test-srv,ct-woodpecker, andct-malformed, and an end-to-end environment for testingct-woodpeckerand Prometheus alerts backed by its metrics.The provided
docker-compose.ymlcan be used to create act-test-srvcontainer providing two certificate transparency logslog-oneandlog-two. Act-woodpeckercontainer is configured to monitor both logs storing the data in the mounted sqlite3 dbstorage.sqlite.A
prometheuscontainer is configured to scrape thect-woodpeckercontainer metrics, and to use analertmanagercontainer to process any alerts. For now thealertmanagercontainer uses an/echoendpoint on one of thect-test-srvlogs as a webhook POST target so that alerts are printed to stdout via thect-test-srv. Theprometheuscontainer comes configured with the alerts from https://github.com/letsencrypt/ct-woodpecker/pull/70 as a starting-point (thanks @pgporada!)To allow testing alert rules the
ct-test-srvnow has a way to add generic mock responses for any path. This can be used (for e.g.) to configure one of the logs to return a 500 error for/ct/v1/add-chainfor a period of time.To use the docker environment, first make sure you have a
storage.sqlitedb created in the project root:rm storage.sqlite 2>/dev/null ; cat storage/schema.sqlite | sqlite3 ./storage.sqliteThen start the containers:
docker-compose upYou can access Prometheus at:
http://localhost:9090You can break certificate submission for
log-twoby running:curl -X POST -d '{"path":"/ct/v1/add-chain","code":404,"response":{"error":"oh noes!"}}' localhost:4601/add-mockShortly afterwards (2-4m) you can expect the
CertSubmissionErrorsalert to be firing inhttp://localhost:9090/alertsbased on thect-woodpeckercontainer being unable to submit certificates tolog-two.You can cause the alert to recover by fixing
log-two's certificate submission by running:curl -X POST -d '{"path":"/ct/v1/add-chain"}' localhost:4601/clear-mockPrometheus scrape data will be maintained in Docker volume called
ct-woodpecker_prometheus_data. If you remove your docker-compose containers make sure to alsodocker volume rm ct-woodpecker_prometheus_datato reset the Prometheus data volume.