GitHub audit tool

[XAMPPRocky]

At the last meeting we discussed the desire to have a tool to be able to audit that a GitHub organisation matches a certain criteria. I think we should make the tool general so that other organisations can use it. The tool should probably take in a config file (or defaults if empty) that display the following warnings if toggled and present. This a tracking issue for tracking the progress

cc @pietroalbini

Warnings

• [x] The organisation doesn't enforce 2 factor authentication.
• [x] The organisation's admins have commit activity.
  • We want to encourage people to have separate accounts for administration and their code activity.
• [x] Repositories in the organisation that don't have a protected master branch.
• [ ] If the owner is not a certain account.
• [ ] If an admin account is doesn't match a whitelist.
• [ ] If an organisation's installed app's doesn't match a whitelist.
• [ ] Membership doesn't match a whitelist.

[XAMPPRocky]

Mozilla also has documentation on guidelines for security for repositories that would be helpful to match. https://wiki.mozilla.org/GitHub/Repository_Security

[pietroalbini]

Just as a personal preference, I think it would be better to use environment variables instead of a configuration file to toggle lints, as it's way easier to deploy the tool (everyone can use the same docker image instead of creating a custom one with the configuration file).

We also need to figure out how we want the tool to report warnings: creating/closing issues on a GitHub repo, pushing metrics and alerts to Prometheus, or emails?

[XAMPPRocky]

Well I've written it as a library + CLI for now since that's how I'm testing it. It should be easy enough to extend this. I do plan to add both a config file and environment variables. I've written an initial prototype here, it audits the 2FA authentication, and whether the admins have had recent push activity. https://github.com/XAMPPRocky/gh-auditor Here's an example of the current output.

⏳ Auditing 2 Factor Authentication
❗️ Warning:
2 Factor Authentication is not required for members of the organisation.

💡 Recommendation:
Enable 2 Factor as a requirement for members.

⏳ Auditing Admin Commit Activity
❗️ Warning:
Admins (XAMPPRocky) have push activity. This is usually an indication that admin members are using their accounts for purposes other than administration.

💡 Recommendation:
Create seperate accounts for administration access to the organisation.