developers.democracyclub.org.uk API

About

This project provides an API gateway in front of other Democracy Club APIs.

• The /postcode and /address APIs provide a nice abstraction over calling both the WDIV and WCIVF APIs and combine the responses.
• The /elections endpoint is a straight proxy for EE's /elections endpoint.
• The project hosts user-facing documentation for the API and sandbox outputs to help data consumers develop applications when real data is not available.
• As time goes on, developers.democracyclub.org.uk should become a one-stop-shop for DC data, additionally proxying additional EE/YNR endpoints.

Local Development

Application

• cp aggregator-api/aggregator/settings/local.example.py aggregator-api/aggregator/settings/local.py

• Install Python dependencies

  • This project makes use of the new pipenv categories. This means a straight pipenv install no longer works, and instead we must install by category. pipenv install --categories "frontend api dev-packages" will get you everything.
  • When installing a new dependency, you must define the category you're installing it into, e.g. pipenv install --categories "frontend" django
  • When removing a dependency you must remove it from the category it's defined in, e.g. pipenv uninstall --categories "frontend" django

• Run the test suite: pytest

• Run lint checks: pytest --ruff

• Auto-format: ruff format .

Documentation

TODO: Update this

Configuration

Local Settings

For local development, the only required setting is SECRET_KEY. In local.py, it may also be useful to set:

• EE_BASE_URL - e.g: to run against a local WDIV install instead of elections.democracyclub.org.uk
• WCIVF_BASE_URL - e.g: to run against a local WDIV install instead of whocanivotefor.co.uk
• WDIV_BASE_URL - e.g: to run against a local WDIV install instead of wheredoivote.co.uk
• WDIV_API_KEY - WDIV API key. In local dev, anonymous access is generally sufficient. You are unlikely to exceed the rate limit.

Production Settings

In production, settings are obtained from environment variables. In production, we need:

• SECRET_KEY - Django secret key.
• WDIV_API_KEY - WDIV API key. For production usage, we must call WDIV with an API key to ensure we are not rate limited.
• SENTRY_DSN - If set, exceptions will be logged here.
• ENV - "prod" or "dev" - used by and sentry. FIXME: not currently propagated via the SAM CLI

Deployment

• The API is hosted on AWS Lambda and built with the AWS SAM CLI
• Deployment of 3 parallel environments is handled by Circle CI
• If you merge to master and all the tests pass, deployments will be triggered:
  • to development and staging in parallel
  • to production, if staging smoke tests pass

Development deployments to AWS Lambda

Development deployments are described in detail in a separate document.

Here are the happy-path steps to create a deployment in DC's development AWS account.

Before following these steps:

• Choose a deployment name and use it wherever these steps mention <NAME>
• Install and configure the local development prerequisites
• Activate the Python virtualenv that contains the local development prerequisites you installed in the previous step

NEW_ENV_NAME=<NAME> python samconfig.toml.d/new-dev-env.py >>samconfig.toml
AWS_DEFAULT_REGION=eu-west-2 pipenv run sam validate
make all
sam build  --config-env <NAME> --use-container --cached
sam deploy --config-env <NAME>

These steps should have deployed the app to Lambda, accessible via AWS API Gateway on the 'AggregatorApiFqdn' domain mentioned near the end of the deployment output, but only when accessed the path '/Prod'.

You can continue and add TLS, caching, and a custom domain to this deployment by following the rest of the deployment document.