search

ossf / scorecard

OpenSSF Scorecard - Security health metrics for Open Source
https://scorecard.dev
Apache License 2.0
4.55k stars 496 forks source link

Feature: maintainer annotation #1907

Open laurentsimon opened 2 years ago

laurentsimon commented 2 years ago

Certain maintainers disagree or are unable to satisfy certain Scorecard checks. It would be useful to provide a way for them to explain their reasoning, if they want. We can surface this in the results next to each non-compliant alert.

Ideally this would be an inlined comment, similar to linter comments - in Go we use // nolinter, for example.

In some cases, it may require a config file (like the OSSF SECURITY-INSIGHT.md). But I'd like to see if we can start with inlined "annotations", because this makes it easier to keep the annotation in sync with the code.

Immediate use case: OSS-Fuzz cannot be pinned by hash because of the way it's setup.

Note: the SECURITY-INSIGHT.md is more appropriate for repo-wide insight, like "my unit tests are stored in this folder". (I'm unable to find the repo for it)

evverx commented 1 year ago

It would be useful to provide a way for them to explain their reasoning, if they want. ... OSS-Fuzz cannot be pinned by hash because of the way it's setup

I wonder why maintainers should waste their time explaining their reasoning? I kind of get that to avoid wasting even more time on receiving bogus promotional PRs triggered semi-automatically (and fixing imaginary "security" issues) it would probably be easier to mark CIFuzz/CFLite as false positives once and for all but I'm not sure why all the options boil down to wasting maintainers' time instead of choosing sensible defaults.

eddie-knight commented 1 year ago

I really like how CLOMonitor displays the justification for exempt checks:

image

evverx commented 1 year ago

Looking at https://clomonitor.io/docs/topics/checks/#exemptions it seems CLOMonitor can't disable checks partially in the sense that if it supported the "Pinned-Dependencies" check it would only be possible to turn it off entirely instead of excluding CIFuzz and CFLite. Even if it was possible to mark only those two actions someone would have to spend their time on adding those exemptions to every repository where those actions are used and I think that would be a waste of time as well.

Then again, I don't care about those false positives much but when those false positives turn into semi-automated PRs wasting maintainers time I don't think it's useful. In the case of CFLite those PRs actually can be considered malicious because they pin the action only without the docker images, which can lead to potentially half-broken fuzzing infrastructure and that isn't ideal because it's used to make sure that patches fixing buffer overflows, UAFs and stuff like that are backported correctly.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity.

github-actions[bot] commented 10 months ago

This issue is stale because it has been open for 60 days with no activity.

github-actions[bot] commented 6 months ago

This issue has been marked stale because it has been open for 60 days with no activity.

ia0 commented 2 months ago

What is the status of this feature? Is it only available in the CLI with the --show-annotations flag? Is it planned to also be supported in the GitHub Action and visible from securityscorecards.dev/viewer?