An app to host content and forms for the "Get Help With Tech" COVID-19 response initiative.
There are two ways to perform a release -
To perform manual releases, you will need:
bundle install
to install the gem dependenciesyarn
to install node dependenciesbin/rails db:setup
to set up the database development and test schemas, and seed with test databundle exec rails server
to launch the app on http://localhost:3000./bin/webpack-dev-server
in a separate shell for faster compilation of assetsThese are the tasks that you will need to run to set up your local db:
bundle exec rails db:seed
to insert mobile network operatorsbundle exec rails import:all
to import schools, responsible bodies and some personas accounts to test with. (NOTE: This can take some time to run but you will have feedback in terminal)Test – pretends to send messages
echo "GHWT__GOVUK_NOTIFY__API_KEY: YOUR_API_KEY_GOES_HERE" > .env
so the local app uses this new keyOpen up a rails console with...
bundle exec rails c`
# creates a single support user
CreateAdminUsersService.new('first.last@digital.education.gov.uk').create!
# or (:support is default)
CreateAdminUsersService.new('first.last@digital.education.gov.uk', :support).create!
# creates a single computacenter user
CreateAdminUsersService.new('first.last@computacenter.com', :supplier).create!
You can only create multiple users of the same type in one batch.
# creates multiple support users
CreateAdminUsersService.new(['first1.last1@digital.education.gov.uk', 'first2.last2@digital.education.gov.uk']).create!
# or (:support is default)
CreateAdminUsersService.new(['first1.last1@digital.education.gov.uk', 'first2.last2@digital.education.gov.uk'], :support).create!
# creates muliple computacenter users
CreateAdminUsersService.new(['first1.last1@computacenter.com', 'first2.last2@computacenter.com'], :supplier).create!
bundle exec rake parallel:setup
bundle exec parallel_rspec spec/models # to run spec directories (or individual spec files) in parallel
bundle exec rake # runs rspec (in parallel) and other code checks
It's best to lint just your app directories and not those belonging to the framework, e.g.
bundle exec rubocop app config db lib spec Gemfile --format clang -a
or
bundle exec scss-lint app/webpacker/styles
bundle exec brakeman
All the above are run automatically on GitHub Actions when pushing a PR.
The rails-erd
gem uses Graphviz to draw diagrams of the ActiveRecord models and their relationships.
You can run it at any time using:
bundle exec rails erd
GOV.UK Notify for sending emails
Computacenter TechSource - this app will post cap update requests to TechSource when we change the number of devices allocated to a school.
In the normal flow of things, the simplest way to perform a release is by using the Github Actions we have set up on this repository.
Merging a Pull Request to main
will automatically run the tests and perform a release to the 'dev' environment.
Once this has completed (you can check the currently deployed commit SHA from http://get-help-with-tech-dev.london.cloudapps.digital/healthcheck.json, and make sure it matches 'main'), you can then promote that image from dev
to staging
, and then from staging
to prod
as follows:
In the blue header, click 'Run workflow', fill in the from and to environments and then click the green 'Run workflow' button (see screenshot below)
CF_DOCKER_USERNAME
and CF_DOCKER_PASSWORD
set to the values of your Docker Hub accountdocker login
to log in to Docker Hubmake dev release
to build, push and deploy the Docker image to GOV.UK PaaS development instancemake staging promote FROM=dev
to deploy the -dev image to stagingmake prod promote FROM=staging
to deploy to productionThe app should be available at https://get-help-with-tech.education.gov.uk/
Sometimes releases fail on production - sometimes this is due to a migration which fails due to unexpected data in production, for instance. When this happens, you'll probably want to minimize user impact by rolling back the release to a previous version ASAP, while you investigate offline.
There are some make tasks that can help:
make prod remote-docker-tags
- list all available tags for production Docker image, on Docker Hub (you can replace prod
with another environment)
make prod rollback-to TAG=...
- re-deploy the Docker image tagged with the given TAG
The normal release process works as follows:
get-help-with-tech-(env name)
dfedigital/get-help-with-tech-(env name):latest
. Re-tag it with replaced-at-(timestamp)
and push it back up to Docker Hub.dfedigital/get-help-with-tech-(env name):latest
, and push it to Docker Hub, overwriting any existing image with the same name and tagdfedigital/get-help-with-tech-(env name):latest
from Docker Hub, and deploy it to the app called get-help-with-tech-(env name)
If you're not building a new image but promoting an image from another environment, the process is a little simpler, but largely the same:
dfedigital/get-help-with-tech-(env name):latest
. Re-tag it with replaced-at-(timestamp)
and push it back up to Docker Hub.dfedigital/get-help-with-tech-(FROM env name):latest
, re-tag it as dfedigital/get-help-with-tech-(TO env name):latest
' and push it back up to Docker Hubdfedigital/get-help-with-tech-(env name):latest
from Docker Hub, and deploy it to the app called get-help-with-tech-(env name)
In either case, before replacing the dfedigital/get-help-with-tech-(env name):latest
image on Docker Hub, it will automatically re-tag the existing image as replaced-at-(timestamp)
- making it straightforward to rollback to any available previous tag.
Over time, the timestamped tags will accumulate in Docker Hub, and should be pruned occasionally. You can do this by visiting https://hub.docker.com/repository/registry-1.docker.io/dfedigital/get-help-with-tech-dev/tags as a logged-in member of the dfedigital
organisation, clicking the checkboxes next to any no-longer-required tags, and choosing 'Delete' from the select box at the top of the list.
Some values are configurable with environment variables:
Name | Description | Default |
---|---|---|
GHWT__SIGN_IN_TOKEN_TTL_SECONDS | Sign-in tokens will expire after this many seconds | 600 |
GHWT__GOVUK_NOTIFY__API_KEY | API key for the GOV.UK Notify service, used for sending emails | REQUIRED |
GHWT__HOSTNAME_FOR_URLS | Hostname used for generating URLs in emails | http://localhost:3000/ |
GHWTHUAWEIDEVICES__PASSWORD | Password for Huawei routers | secret |
GHWT__GOVUK_NOTIFYTEMPLATESSIGN_IN_TOKEN_MAIL | ID of the template in GOV.UK Notify used for mailing sign-in tokens | '89b4abbb-0f01-4546-bf30-f88db5e0ae3c' |
GHWT__STATIC_FILE_CACHE_TTL | how long CDNs and browsers should cache static assets for in production, in seconds. | (nil) |
GHWTTHROTTLE* | Request throttling limits, see settings.yaml for more info | (see settings) |
GHWTLOGSTASHHOST | Hostname for where logstash should send logs | (nil) |
GHWTLOGSTASHPORT | Port for where logstash should send logs | (nil) |
GHWTSENTRYDSN | DSN (Client key) for Sentry.io error reporting | (nil) |
GHWTCOMPUTACENTEROUTGOING_API__ENDPOINT | URL of the CapUpdateRequest API at TechSource | (nil) |
GHWTCOMPUTACENTEROUTGOING_API__USERNAME | Basic auth username to use for the TechSource CapUpdateRequest API | (nil) |
GHWTCOMPUTACENTEROUTGOING_API__PASSWORD | Basic auth password to use for the TechSource CapUpdateRequest API | (nil) |
GHWTZENDESKUSERNAME | Username for Zendesk account to be able to use the Zendesk API. Both Zendesk options need to set before Zendesk API can be used. | (nil) |
GHWTZENDESKTOKEN | Token for Zendesk account to be able to use the Zendesk API.Both Zendesk options need to set before Zendesk API can be used. | (nil) |
GHWTDATABASE_FIELD_ENCRYPTIONKEY | Secret key for the encrytion fields in the assets table | REQUIRED |
GHWTDATABASE_FIELD_ENCRYPTIONSALT | Salt for the encryption field in the assets table | REQUIRED |
GHWT__SLUG_CHECKSUM_SECRET | A secret for hashing a checksum for IDs in the URL | REQUIRED |
GHWT__API_TOKEN_TTL | TTL in days after which API Tokens CC use will expire from creation date | 90 |
GHWT__SITE_BANNER_MESSAGE | Banner message text to appear at the top of all pages of the site | (nil) |
GHWT__LONG_FORM_SITE_BANNER_MESSAGE_FLAG | Boolean value, when true, displays the long form banner message text to appear at the top of all pages of the site from somefile.md | false |
GHWT__LONG_FORM_SITE_BANNER_MESSAGE_PARTIAL | Path to the markdown partial file containing the long form site banner message. Default resolves to site_banner/_long_form.md | site_banner/long_form |
See the settings.yaml file for full details on configurable options.
Certain aspects of app behaviour are governed by a minimal implementation of Feature Flags. These are activated by having an environment variable FEATURES_(flag name) set to 'active', for example:
# start the rails server with rate limiting active
FEATURES_rate_limiting=active bundle exec rails s
The available flags are listed in app/services/feature_flag.rb
, and available in the constant FeatureFlag::FEATURES
.
Name | Description |
---|---|
FEATURES_gias_data_stage_pause | Pauses any GIAS data staging of trusts and schools, should be used e.g. in Septemeber when GIAS data is changed a lot and unsettled |
To display, set and unset feature flags on GOV.UK PaaS:
# display the feature flags
cf env (app name) | grep FEATURES
# For example:
cf env get-help-with-tech-prod | grep FEATURES
# set an env var
cf set-env (app name) (environment variable name) (value)
# For example:
cf set-env get-help-with-tech-prod FEATURES_rate_limiting active
# To unset the var:
cf unset-env (app name) (environment variable name)
# For example:
cf unset-env get-help-with-tech-prod FEATURES_rate_limiting
Log on to the first instance of the sidekiq container with:
make (env) ssh
If you want to ssh to a particular process and/or particular instance, you can be specific as follows:
make (env) ssh PROCESS=(web|sidekiq) INSTANCE=(0|1|...)
Both parameters are optional. PROCESS defaults to sidekiq, to prevent any possibility of using too much memory in the console and accidentally getting the web container restarted by the hosts' monitoring. INSTANCE defaults to 0.
Some service steps can only be carried out using the Rails console. To get to the console on GOV.UK PaaS, first ssh
as above.
Once you have a prompt on the container, you'll be in a subshell with the Rails app's environment variables, and in the app's root directory:
In this subshell, you can then launch the console in the normal way:
bundle exec rails c
Note that the log messages you see will be in structured JSON format, for aggregation to Kibana via Logstash.
For clarity, you might want to unset RAILS_LOG_TO_STDOUT
before launching the console.
Files can be uploaded/downloaded as follows:
make (env) upload LOCAL_PATH=/some/local/file/path REMOTE_PATH=/some/remote/path PROCESS=(web|sidekiq) INSTANCE=(0|1|...)
make (env) download LOCAL_PATH=/some/local/file/path REMOTE_PATH=/some/remote/path PROCESS=(web|sidekiq) INSTANCE=(0|1|...)
The PROCESS and INSTANCE parameters follow the same conventions as for make (env) ssh
- defaulting to instance 0 of the sidekiq process.
You will be prompted for a password for the connection - this is a one-time-only password, generated on-demand via the GOV.UK PaaS platform. The output just above the prompt will tell you what the password is for this session - you should see a message like this:
Connecting to 'sidekiq' instance '0'
*** Enter (some hex string) at the password prompt (this is a one-time-only password) ***
scp -P 2222 -o StrictHostKeyChecking=no -o User=cf:(...) ssh.london.cloud.service.gov.uk:/tmp/test.txt /tmp/
cf:(....)@ssh.london.cloud.service.gov.uk's password:
Just copy-and-paste the given password into the prompt, and the transfer will begin.
Tail the logs for a given env:
make (env) logs
View recent logs for a given env:
make (env) logs-recent
Semantic Logger is used to generate
single-line logs. In production environments, when RAILS_LOG_TO_STDOUT
is
enabled, this is configured to output JSON logs. These logs are then sent to the
log aggregator.
You can configure logstash to send logs to your log aggregator by setting the logstash host and port environment variables.
A copy of the logstash filter we use exists in the repo. This has to be installed manually in the log aggregator to be used.
If you need to amend the Get help with technology: devices guidance:
Make your amends:
Get a colleague to review your pull request – the change cannot be merged without this step
Once the pull request is merged, it will be released the next time the service is released.