oxsecurity / megalinter

πŸ¦™ MegaLinter analyzes 50 languages, 22 formats, 21 tooling formats, excessive copy-pastes, spelling mistakes and security issues in your repository sources with a GitHub Action, other CI tools or locally.
https://megalinter.io
GNU Affero General Public License v3.0
1.81k stars 215 forks source link

Trufflehog as a GitHub Action fails at .git/config #3046

Open wesley-dean-flexion opened 8 months ago

wesley-dean-flexion commented 8 months ago

Describe the bug

This is a continuation of #2834 .

I'm running MegaLinter v7.4.0 (Python flavor) as a GitHub Action. When it runs, it finds .git/config (which isn't in the repository as it's created in a previous step (probably actions/checkout@v4)), potentially in conjunction with the inclusion of a PAT environment variable (so applied fixes can be pushed back). While the finding is "correct," it's a result of the expected workflow.

Here's some sample output:

βœ… Linted [REPOSITORY] files with [trufflehog]: Found 1 non blocking error(s) - (4.4s) (expand for details)
  - Using [trufflehog v3.60.4] https://megalinter.io/7.3.0/descriptors/repository_trufflehog
  - MegaLinter key: [REPOSITORY_TRUFFLEHOG]
  - Rules config: identified by [trufflehog]
  --Error detail:
  πŸ·πŸ”‘πŸ·  TruffleHog. Unearth your secrets. πŸ·πŸ”‘πŸ·

  Found verified result πŸ·πŸ”‘
  Detector Type: Github
  Decoder Type: BASE64
  Raw result: ***
  Site_admin: false
  Name: ***
  Company: 
  Rotation_guide: https://howtorotate.com/docs/tutorials/github/
  Username: ***
  Url: ***
  Account_type: User
  File: .git/config
  Line: 11

What's interesting here is that the result was verified, so the --only-verified flag mentioned as a workaround wasn't... working around.

The --exclude-paths flag to Trufflehog accepts a filename that contains a list of paths to ignore -- one can't just tell it to exclude .git . Using --exclude-paths=.gitignore doesn't work in our particular situation on account of our .gitignore containing some fun fileglobbing that --exclude-paths doesn't understand.

I was able to workaround by creating a separate file, .trufflehogignore and adding .git/ to it, then telling Trufflehog to ignore paths from that file. From the project's .mega-linter.yml:

REPOSITORY_TRUFFLEHOG_ARGUMENTS: "--exclude-paths=.trufflehogignore"

To Reproduce Steps to reproduce the behavior:

  1. Invoke Megalinter via GitHub Action
  2. Include a step that results in the creation of a .git/ directory
  3. Run Megalinter on a commit
  4. See error

Expected behavior

Trufflehog, run through Megalinter, should not throw a non-blocking error solely as a consequence of using a checkout action (which creates the file that's being matched)

Here's the result of running Megalinter with the applied workaround:

βœ… Linted [REPOSITORY] files with [trufflehog] successfully - (3.62s) (expand for details)
  - Using [trufflehog v3.60.4] https://megalinter.io/7.4.0/descriptors/repository_trufflehog
  - MegaLinter key: [REPOSITORY_TRUFFLEHOG]
  - Rules config: identified by [trufflehog]
wesley-dean-flexion commented 8 months ago

Pedantically speaking, I don't know if this is a Megalinter bug. We could script around it (e.g., create a file at runtime that tells Trufflehog to ignore .git and pass that file along to Trufflehog), doing so could be problematic for other uses where there may be other --exclude-paths options passed along. That is, I don't know if Trufflehog will respect the inclusion of multiple files or if it'll fail with an error (like it does with --skip-unverified).

Therefore, my hope is that should someone else stumble upon this issue and search the list of Megalinter issues, they may find this and it may help them work around the issue with less trial and error.

Feel free to close this out as-needed.

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale πŸ€– label or comment on the issue.

pjungermann commented 4 months ago

We did the same change with the same naming, ignoring .git/ as well.