🕵️ Identify Owners of GitHub Repository Estate

ministryofjustice / operations-engineering

This repository is home to the Operations Engineering's tools and utilities for managing, monitoring, and optimising software development processes at the Ministry of Justice. • This repository is defined and managed in Terraform

MIT License

11 stars 5 forks source link

User Need

As a Engineering Leader I want to identify all GitHub Repositories in my estate so that I can monitor my GitHub Repository estate effectively

Value

To increase visibility to Engineering Leaders of their GitHub Repository estate.

To increase the visibility of asset owners when they need to be identified (primarily by Operations and Security).

Functional Requirements (What):

[ ] Process can be triggered manually ad-hoc
[ ] Process identifies owners by a GitHub name prefix
[ ] Process identifies owners by Admin access via a GitHub Team
[ ] Process identifies owners by topic

Non-Functional Requirements (How):

[ ] Results are easy to understand

Acceptance Criteria:

[x] Percentage of GitHub Repository estate (roughly) owned by HMPPS
[x] Percentage of GitHub Repository estate (roughly) owned by LAA
[x] Percentage of GitHub Repository estate (roughly) owned by Central Digital
[x] Percentage of GitHub Repository estate (roughly) owned by OPG
[x] Percentage of GitHub Repository estate (roughly) owned by Platforms and Architecture
[x] Percentage of GitHub Repository estate (roughly) unowned
[x] Other teams/business units identified that may be part of the unowned estate

Notes:

We are currently talking to Engineering Leaders every month to better co-ordinate and apply central and localised standards. As part of this, I agreed to complete an initial poc of gathering estate data based on identification strategies provided by the leads. These strategies mainly included:
- Identify by admin access via a GitHub Team (hierarchical)
- Identify by prefix
- Identify by topic
The exact details of the above strategies, for each "owner" should be discussed with the leads in that particular area. Although this ticket is for an initial run at this, so it doesn't have to be perfect.

🆙 Update

Some numbers from the GitHub Repository Owner work 👀 The data is for Public and Active repositories and the rules being used are on this PR 📏

{
"totals": {
"totalRepoCount": 854,
"hmppsRepoCount": 361,
"laaRepoCount": 71,
"opgRepoCount": 62,
"centralDigitalRepoCount": 59,
"platformsAndArchitectureRepoCount": 178,
"unownedRepoCount": 186,
"totalReposWithFoundOwners": 668,
"reposWithMultipleOwnersCount": 33
},
"percentages": {
"hmppsRepoCount": "42%",
"laaRepoCount": "8%",
"opgRepoCount": "7%",
"centralDigitalRepoCount": "7%",
"platformsAndArchitectureRepoCount": "21%",
"unownedRepoCount": "22%",
"totalReposWithFoundOwners": "78%",
"reposWithMultipleOwnersCount": "4%"
}
}

I've gone through the unowned estate and the repositories are too small in groupings to add any more rules at a high level

📝 Notes

It is worth noting that the current ruleset is loosely based on conversations with Engineering Leads and some of my own assumptions after looking at the data, the rules most likely need more refinement -but will do to spark conversations ⚡️

The repos with multiple owners mainly consist of Platform and Architecture Repos where multiple teams are added to allow everyone to edit the repositories. This is to be expected.

🆙 Update

Added functionality to fetch all parents of a team attached to a repository

Result for search against Public and Active Repositories in the ministryofjustice organisation where

A pre-defined GitHub Team had Direct Administrator Access
OR

A sub-team had Direct Administrator Access:

{
"totals": {
"totalRepoCount": 853,
"hmppsRepoCount": 271,
"laaRepoCount": 36,
"opgRepoCount": 59,
"cicaRepoCount": 1,
"centralDigitalRepoCount": 32,
"platformsAndArchitectureRepoCount": 241,
"unownedRepoCount": 195,
"totalReposWithFoundOwners": 658,
"reposWithMultipleOwnersCount": 9
},
"percentages": {
"hmppsRepoCount": "32%",
"laaRepoCount": "4%",
"opgRepoCount": "7%",
"cicaRepoCount": "0%",
"centralDigitalRepoCount": "4%",
"platformsAndArchitectureRepoCount": "28%",
"techServicesRepoCount": "3%",
"unownedRepoCount": "23%",
"totalReposWithFoundOwners": "77%",
"reposWithMultipleOwnersCount": "1%"
},
}

I then extracted the unknown estate and ran a further fuzzy match on those 195 repositories where

A pre-defined GitHub Team had Direct Access (any level)
OR
A sub-team had Direct Any Direct Access (any level)
OR

Repository name had pre-defined prefix


{
"totals": {
"totalRepoCount": 195,
"hmppsRepoCount": 60,
"laaRepoCount": 40,
"opgRepoCount": 1,
"cicaRepoCount": 3,
"centralDigitalRepoCount": 1,
"platformsAndArchitectureRepoCount": 12,
"unownedRepoCount": 76,
"totalReposWithFoundOwners": 119,
"reposWithMultipleOwnersCount": 0
},
"percentages": {
"hmppsRepoCount": "31%",
"laaRepoCount": "21%",
"opgRepoCount": "1%",
"cicaRepoCount": "2%",
"centralDigitalRepoCount": "1%",
"platformsAndArchitectureRepoCount": "6%",
"techServicesRepoCount": "1%",
"unownedRepoCount": "39%",
"totalReposWithFoundOwners": "61%",
"reposWithMultipleOwnersCount": "0%"
},

}



## 👀 Remaining Work

- No remaining work on this ticket, but the code, rules and results will be discussed with the lead engineers of each area

## 📝 Notes

- P&A has probably the largest repository estate if we include the moj-analytical service organisation
- The queries don't include private and internal repositories which may offer additional interesting insights

ministryofjustice / operations-engineering