Scan git repos for sensitive data

[sid-secops]

Background To scan MoJ git repos to prevent credentials and sensitive data being exposed. aws access keys, docker passwords, user credentials, sql db backup dump etc https://help.github.com/articles/removing-sensitive-data-from-a-repository/

Approach

• Check the below repo scanning tools
• The repo scan tool should run against the repo every time there is a Pull
• Request to merge to master branch
• Create a wrapper script so that it runs on a frequency every day

Check the repositories https://github.com/ministryofjustice/repo-audit https://github.com/michenriksen/gitrob https://github.com/zricethezav/gitleaks

Prevent secrets from being committed to the repos https://github.com/awslabs/git-secrets

Detecting secrets in source code https://auth0.engineering/detecting-secrets-in-source-code-bd63b0fe4921 https://github.com/auth0/repo-supervisor https://github.com/fugue/credstash

Definition of Done

• The scan tool should be able to scan all the MoJ git repos for sensitive data
• Identify the product owner for each repo
• The scan tool should run on a desired frequency every day (example - every hour?)
• Alert the team on slack or email
• Identify alert response steps - Identify the severity level for the leak, make the repo private within MoJ, lock down the permissions for the repo, contact the service/product owner of the repos for Root Cause analysis

Stretch Goal -

• To check for sensitive data by embedding the scan tool in the CI/CD pipeline https://github.com/zricethezav/gitleaks-ci
• Setting a Jenkins job guarding a repository http://ebi-cloud-portal.readthedocs.io/en/latest/securing_credentials.html

Other References -

• A bash-tool to store your private data inside a git repository. http://git-secret.io https://github.com/sobolevn/git-secret
• Using awslabs/git-secrets https://cloudonaut.io/avoid-security-credentials-on-github/

[ghost]

The two apprentices Aaron and Ashley have coding tasks to start next week. Please get them involved in writing the wrapper script for the scan tool and automation thereof

[kalbir]

I've changed the description based on the planning discussions we had. I'm not sure who @8032 is but if they want to have a chat about getting the apprentices involved that would be helpful.

[sid-secops]

Hi Kalbir, Between the two guys, one of them will pair for the task. I will update who will be assigned. thank you very much for creating the new card with break down of tasks

[sid-secops]

This has been completed - https://waffle.io/ministryofjustice/cloud-platform/cards/5b2b792be28cfc0032f9c9dd