ossf / scorecard

OpenSSF Scorecard - Security health metrics for Open Source
https://scorecard.dev
Apache License 2.0
4.46k stars 489 forks source link

:bug: fix Docker remediations for unpinned GHA dependencies #4131

Closed raboof closed 4 months ago

raboof commented 4 months ago

What kind of change does this PR introduce?

Bug fix

What is the current behavior?

As both the check for unpinned dependencies in GitHub Actions and the check for unpinned Docker dependencies contribute to d.Dependencies, the loop that created remediations for Docker dependencies would also create try to create Docker remediations for the unpinned GitHub Actions dependencies.

This could get really slow, especially when scanning a repo with many GitHub Actions such as https://github.com/apache/beam.

What is the new behavior (if this is a feature change)?**

Only create Docker remediations for Docker dependencies (and similar for GitHub workflow remediations)

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to the release-note

(In particular, describe what changes users might need to make in their application as a result of this pull request.)

Improve Pinned-Dependencies remediation creation
spencerschrock commented 4 months ago

/scdiff generate Pinned-Dependencies

github-actions[bot] commented 4 months ago

Here's a link to the scdiff run

raboof commented 4 months ago

How did the speed-up look on your end?

scorecard --repo=https://github.com/apache/beam --checks Pinned-Dependencies went from 16 minutes to 24 seconds on my machine ;)

Is there still need for improvement

Faster is always better, and I haven't done any further profiling, but I suspect the bottlenecks will be elsewhere now.

This continues with collectDockerfileInsecureDownloads, collectShellScriptInsecureDownloads, and collectGitHubWorkflowScriptInsecureDownloads.

These don't iterate over the Dependencies, though, as far as I noticed

spencerschrock commented 4 months ago

scorecard --repo=https://github.com/apache/beam --checks Pinned-Dependencies went from 16 minutes to 24 seconds on my machine ;)

Fantastic. Thanks for catching this!

Faster is always better, and I haven't done any further profiling, but I suspect the bottlenecks will be elsewhere now.

This continues with collectDockerfileInsecureDownloads, collectShellScriptInsecureDownloads, and collectGitHubWorkflowScriptInsecureDownloads.

These don't iterate over the Dependencies, though, as far as I noticed

I did a quick and dirty test to avoid going over the GHA deps again in collectDockerfilePinning and got another 20% speedup: 24s to 20s on my machine forgithub.com/apache/beam, which is better but like you said probably no longer a bottleneck.

spencerschrock commented 4 months ago

/scdiff generate Pinned-Dependencies

github-actions[bot] commented 4 months ago

Here's a link to the scdiff run