FIREBREAK: 🔥 Create Find a GitHub Repository Owner Service

[connormaglynn]

👀 Purpose

• To enable easy identification of GitHub Repository Owners
• To enable easy identification of an Owners GitHub Repository estate
• To be able to easily articulate and monitor risk associated with unowned repository estate

✅ Tasks

• [x] ✨ Add a New Repository Called operations-engineering-find-a-github-repository-owner
• [x] ✨ Add a New Cloud Platform Namespace called operations-engineering-find-a-github-repository-owner-dev, with a database (RDS, DynamoDB etc.)
• [x] ✨ Add Scraper Job to Query GitHub and Map Repositories to Owners (example script with mapping logic) and Populate Database
• [x] ✨ Add UI to Search Asset Owners and Retrieve GitHub Repositories
• [x] ✨ Add UI to Search GitHub Repository and Retrieve Asset Owners
• [x] ✨ Add UI to Display Asset Owner Metrics i.e. Percentage of Estate Owned by X, Percentage of Unowned Estate

📓 Notes

• An overview of the system, displaying the initial MvP 👇

• The Database Schema should be easily searchable by Owners since retrieving this data takes a while. So an MVP schema may look something like 👇 Mapping a repository to its relevant owners is far less expensive query, so can be done on the fly for the MvP 🚀 Keeping the schema one directional at the beginning will make adding records significantly easier

• Overtime, we may opt to amend the database schema so that it's easily traversable in both directions, this will make adding records more complex but searching them much easier. The example proposed future schema is below 👇

[connormaglynn]

🆙 Update

• Added an MVP UI deployed at https://find-a-github-repository-owner-dev.cloud-platform.service.justice.gov.uk/, which:
  • Displays owner analytics via a Pie Chart
  • Allows searching via repository name

👀 Remaining Work

• Add the scraper job to retrieve, map and store results in a database
• Add connection from UI to database to read data (instead of using the currently hard-coded results)
• Add filter function by Owners

[connormaglynn]

🆙 Update

• MvP deployed and integrated with Public GitHub Repositories in the Ministry of Justice Organisation. The scraper job runs daily to update the mappings. I will close this ticket now.

MvP Solution

Technical Architecture

👀 Remaining Work

• It would be ideal to filter the data more comprehensively i.e. filter by owner and then search for a repository name, search where there is overlapping owners or where there is only a single owner etc.
• To enable better insight into the data, it would be ideal to store the different relationships between assets and owners i.e. does the owner have administrative access?
• We should add in internal, private and archived repositories to better assess our entire estate
• We should make this data available via an API to be used by other services such as Repository Standards to enable the enrichment of data sets
• We should create a production environment for the service (the development environment could hook into the ministryofjustice-test organisation instead)
• We should add an authentication layer to the service to restrict public access to the data