trufflesecurity / trufflehog

Find and verify secrets
https://trufflesecurity.com
GNU Affero General Public License v3.0
14.39k stars 1.57k forks source link

Seperate metadata from code secret scanning. #1892

Open CameronLonsdale opened 8 months ago

CameronLonsdale commented 8 months ago

Description

I'm a big fan of the new functionality to scan issues and comments for secrets, however it's currently not possible to only scan those sources, and ignore cloning & scanning the code in the repository. For maximum flexibility, it would be great to separate each source so you can opt-in (or out) of cloning & scanning the repo & only doing the issues & comments.

Preferred Solution

opt-in or out of cloning & scanning repository files when also scanning repository issues & comments.

ahrav commented 8 months ago

That's an interesting idea @CameronLonsdale , could you elaborate a bit more on your specific use case? Is it so you don't have to clone the source code and simply rely on scanning the comments/issues, or is the full scan just not fast enough?

CameronLonsdale commented 8 months ago

@ahrav Either I tend to already have the repo cloned (so no need to clone it again), or I prefer to have the cloning step be done ahead of time due to complexities with setting it up, which was tricky to debug through trufflehog so I opted for cloning in a separate step. As a result I use the git mode for scanning, but then unable to use the new flags to scan issues & pr comments for secrets.

rgmz commented 8 months ago

On that note, using trufflehog github with comment scanning enabled makes code scanning much slower because it cannot scan the next repository until scanComments has completed.

Scanning comments ("metadata") is subject to rate-limits, whereas cloning is not. If you hit your rate-limit for comment scanning, you'll need to wait up to 30 minutes before the next repository is scanned despite that not being blocked by the rate-limit. Separating code and comment scanning would allow for more efficient scanning of both.