search

iosifache / semgrep-rules-manager

Manager of third-party sources of Semgrep rules 🗂
https://semgrep.iosifache.me
GNU General Public License v3.0
70 stars 6 forks source link

Dealing with random pipeline yaml files in repos #1

Closed meme-lord closed 1 week ago

meme-lord commented 3 weeks ago

I'm not sure if I'm doing something wrong but if I do

semgrep-rules-manager --dir semgrep-rules/ download

And then go to scan a project

semgrep --config ~/semgrep-rules/ .

I will inevitably get a bunch of errors about yaml files in the repos that are not rules. Is there a flag I need to use or do people just manually delete these non rule files?

What I did to get past the non rule yaml files was to delete them but this doesn't feel like a very clean solution.

rm ~/semgrep-rules/elttam/.pre-commit-config.yaml ~/semgrep-rules/community/.pre-commit-config.yaml ~/semgrep-rules/gitlab/.gitlab-ci.yml
rm -rf ~/semgrep-rules/gitlab/ci/ ~/semgrep-rules/gitlab/qa/ ~/semgrep-rules/trailofbits/.github/ ~/semgrep-rules/community/.github/ ~/semgrep-rules/elttam/.github/
iosifache commented 2 weeks ago

Hi @meme-lord,

Thanks for reporting this!

Indeed - some files from the repositories used as sources contain YAML file which are not valid Semgrep rules (for example, elttam/.pre-commit-config.yaml, which is a pre-commit action).

I already implemented an ignore mechanism in the YAML file with all sources, sources.yaml:

https://github.com/iosifache/semgrep-rules-manager/blob/c27c35248f37b4d4d155df154343d8734c886577/semgrep_rules_manager/data/sources.yaml#L15-L19

The fix here will be to migrate all folders and files names that you used in your rm commands into the corresponding ignored sections of the sources.yaml file. Would you like to contribute to this repository by doing the highlighted modification in a PR?