search

ossf / scorecard

OpenSSF Scorecard - Security health metrics for Open Source
https://scorecard.dev
Apache License 2.0
4.58k stars 497 forks source link

Feature Scorecard badges #271

Closed naveensrinivasan closed 2 years ago

naveensrinivasan commented 3 years ago

Is your feature request related to a problem? Please describe. Scorecard should provide a badge for repositories to include in their README to display their compliance.

Scorecard badges

The scorecard should provide badges similar to other https://github.com/badges/shields OSS badges for compliance.

Goals

Implementaion

In this example

naveensrinivasan commented 3 years ago

@inferno-chromium What are your thoughts on this?

laurentsimon commented 3 years ago

Is there going to be a single badge/score, or one for each check?

naveensrinivasan commented 3 years ago

I would think of a single badge with different levels like CII https://bestpractices.coreinfrastructure.org/en/projects/569

laurentsimon commented 3 years ago

sounds like a good idea. SLSA levels would be great, but I'm not 100% sure they directly map to the scorecard checks.

naveensrinivasan commented 3 years ago

sounds like a good idea. SLSA levels would be great, but I'm not 100% sure they directly map to the scorecard checks.

That is even better IMO. We have to just agree on what check constitutes to the levels.

laurentsimon commented 3 years ago

@mmaraya is driving remediation efforts and working on the definition of 'tiers', aka levels for oss. @mmaraya, feel free to provide some insights/feedback.

inferno-chromium commented 3 years ago

SLSA is wip, and only a set of checks will apply to SLSA not all. So, this needs more thought.

mmaraya commented 3 years ago

I like the idea of a single badge with different levels expressed by the number of checks passed. We'll need to list the applicable checks, some of which could come from SLSA requirements but maybe not the levels themselves.

naveensrinivasan commented 3 years ago

I like the idea of a single badge with different levels expressed by the number of checks passed. We'll need to list the applicable checks, some of which could come from SLSA requirements but maybe not the levels themselves.

I disagree with having checks passed as a number in the badge. It could be sending a wrong message to the consumer of any of the repositories/packages. Also, it would discourage from repositories adopting the badge.

For example, https://deps.dev/go/k8s.io%2Fkubernetes has 9/16of 16 checks only 9 passes. So if I as a user would like to consume the k8s package I would be cautious because the percentage of failure is more than 40%.

But in reality, some of the checks aren't applicable to k8s like signed-releases, SAST, Automatic-Dependency-Update.

Having a tiered approach to badges would be helpful similar to CII/SLSA.

Thoughts?

laurentsimon commented 3 years ago

I also prefer qualitative information. In addition to what Naveen said, it empowers consumers of the dependency to make informed decisions about the risks they are comfortable taking. Some users may weight different checks differently so it's hard for us (scorecard) to make that decision on behalf of everyone.

Let's discuss this more in an upcoming meeting.

mmaraya commented 3 years ago

Agree that this needs more discussion. We'll need to find a balance across badge adoption, check applicability, and giving package users easily-digestible information to make a risk-based decision. Note that I wasn't proposing a percentage as not all checks will be applicable to all packages, so the denominator could vary from package to package.

azeemshaikh38 commented 2 years ago

This is now on our roadmap. @laurentsimon @asraa could one of you update this issue with a high-level overview of the design?

azeemshaikh38 commented 2 years ago

@rohankh532 fyi.

azeemshaikh38 commented 2 years ago

https://github.com/ossf/scorecard-action/issues/133 fixes this. Closing the issue.