Script to mine repositories for Git secrets

[HarshCasper]

Description

Sometimes we inadvertently commit secrets in our GitHub repository that otherwise should not have been there. This creates a lot of risk, especially when we expose proprietary information, like our API Keys, AWS configs and more. To prevent this, we can scan the repositories for these secrets and warn, if we find any. This script should take up GitHub repositories and scan them for any secrets if found.

Ideally this can serve as a CLI tool that can be used in the following ways:

• Scanning personal repositories (Public Repositories only)
• Scanning trending repositories (Fetch using any unofficial GitHub API)
• Scanning repositories from a CSV File (Single-Column with the GitHub Repo URL)

You can use your own customization to find how the script can serve the purpose

Language

• [X] Go
• [X] Javascript
• [X] Python

[rahulraikwar00]

@vybhav72954 I think github it self does this right?...if we accidentally pushed any type of Credentials..it informs us by email but it is still limited for GitHub credentials. I think this is something we can work on What do you think? Have any idea or suggestion?

[vybhav72954]

@vybhav72954 I think github it self does this right?...if we accidentally pushed any type of Credentials..it informs us by email but it is still limited for GitHub credentials. I think this is something we can work on What do you think? Have any idea or suggestion?

Yup GitHub Actively looks for secrets. But only for the Password and the email if it's set to be invisible, What Harsh wants is to scan the Repo for API keys and other Configs as well.

[vybhav72954]

Actually, this issue is a duplicate of #434

I worked in this as well, the max I was able to achieve was look for .env files. I was able to achieve this, but there are a few problems with that approach.

• .gitignore usually ignores .env.
• Those who know about .env are aware enough not to commit it in the first place.

So well my attempt was mostly useless and that's why I never pushed it. In case you have a better approach, we can discuss it here as well.

[HarshCasper]

Here is something that we can use as an inspiration:

• Git Secrets
• Gittyleaks
• Python Git Scripts

Your approach is also good @vybhav72954. Perhaps that solution can serve as the v1 of the script and can be further developed on top of it.