Spike - Investigate What Data is Too Sensitive For Logs

[connormaglynn]

Background 👀 We are currently redirecting logs for some workflows to a file and uploading these as artefacts (Example here.

The impact of this is that logs are not available and only exceptions are only shown in jobs that fail. Further details are provided for logs that succeed since the upload artefact job is then fired. This makes debugging issues much harder.

It's unclear why we are uploading the logs as artefacts, though may be related to concerns about publicising sensitive data/personal data or PII (see slack thread).

Although, from some reading, artefacts provide no additional security provisions over normal logging within the job.

Task 🚀 Discuss with the team what data is considered too sensitive for logs.

There then should be a separate ticket raised to remove all logging with this information and to re-enable normal logging in the pipelines (unless we're making use of artefacts).

Useful 🧠

• This ticket is mainly in reference to two jobs amend-user-repository-access found in the operations-engineering repository for both MoJ and MoJAS orgs
• PDF on Determining Personal Data

Acceptance Criteria 🎯

• The team understands what data is considered too sensitive for logs, preferably this understanding should be documented somewhere.
• A separate ticket is raised if any additional action is needed (i.e. removing data from logs and re-enabling normal logging)

[NickWalt01]

The scripts amend-user-repository-access has been removed from the operations-engineering repository for both MoJ and MoJAS orgs and replaced with the move users to a team app. this is has no logging nor stores the logging to a file.

[NickWalt01]

These repos contain PII:

Public Repos: https://github.com/moj-analytical-services/github-outside-collaborators https://github.com/ministryofjustice/aws-root-account https://github.com/ministryofjustice/github-collaborators

Private Repos: https://github.com/moj-analytical-services/inactive-users https://github.com/ministryofjustice/inactive-users

[NickWalt01]

A non MoJ Org member can read the public repository files to see the PII in the above repo's. They cannot read the print out of a CI action unless they are a member of the GH Org. A Org user would require direct access to the private repositories to see the PII data within the repository files or the CI action trace. The public can see the username of who has committed to a repository, some username like NickWalt01 could easily be matched to the users listed publicly on the https://github.com/orgs/ministryofjustice/people website. It isn't difficult to guess an email address is name.surname@digital.justice.gov.uk using the users GH username and an example of an email as seen in a GH collab repo .tf file. These .tf files provides a username, name, email address of a user and the MoJ email address of the approver for every collaborator. This could all be considered PII information in a work context. The MoJ GH Org is suppose to be open so information like this will be public visibly. It has been like this since 2020. I do not see an issue of using usernames / email addresses of MoJ users and third party users within our repositories for official work purposes. There could be a PII issue if a users personal email address has been provided. The inactive users repo previously used a .csv export from GitHub that has a small chance of containing a personal email addresses when the user hasn't set there work email address as the primary email address linked to their GH account but GH may have blanked out their email address as it was an export from the Org. The .csv file is no longer on the repository and because it is a private repository only our team have access to files and CI action trace. The repo history could be scrubbed to remove references to that file to be sure.

[NickWalt01]

Other project teams are implementing IaC using work email addresses and names.

This document covers PII in logs: https://security-guidance.service.justice.gov.uk/online-identifiers/#online-identifiers-in-security-logging-and-monitoring but covers customers who interact with MoJ rather than internal MoJ users.

A data-security-and-privacy document: https://security-guidance.service.justice.gov.uk/data-security-and-privacy/#data-security-and-privacy

[NickWalt01]

As we are not logging any data to log files in our repo's I am closing the ticket. A potential issue I can see is if any repo were to log a users personal email address via GH / Auth0 / Dockerhub then it would need to be modified so that it doesnt store that data.

CI action traces are deleted by GH after 90 days so previous traces will have been automatically removed already