Rename "CII Best Practices Badge" to "OpenSSF Best Practices Badge"

[david-a-wheeler]

Describe the bug Rename "CII Best Practices Badge" to "OpenSSF Best Practices Badge"; the project recently changed its hame.

[azeemshaikh38]

@naveensrinivasan would you able to tackle this? Can be a P1/P2 item in the current list we have.

[naveensrinivasan]

I can.

[david-a-wheeler]

There are 2 related issues, the name of the metric includes CII, and the description includes CII. I expect the metric rename is a bigger deal, so we could change the description first & then do the actual name change later.

[justaugustus]

Suggestion: Copy/paste the existing check/use the same code path for the new one. Mark the existing as deprecated in one release. Remove in the next release.

[gabibguti]

May I suggest that we also change the check description to make it clear we are verifying if the project completed the OpenSSF Best Practices form? Because, with the current description, it sounds like the check wants you to add a badge to your project. In fact, if you score a 0/10 in this check, Scorecard reports "no badge found", but deep down it looks for the form, not the badge. You might not want to display such badge, so it refrain users from completing this check.

[david-a-wheeler]

May I suggest that we also change the check description to make it clear we are verifying if the project completed the OpenSSF Best Practices form?

It shouldn't. It should determine how fully it meets a badge. You can "fill in a form" and not earn a BP badge. In fact, you can "fill in a form" to show you meet 0 requirements today :-).

Because, with the current description, it sounds like the check wants you to add a badge to your project.

Are you assuming that "badge" is "a graphical image"? That's not what "Best Practices Badge" means. In particular, the current text asks if you have earned a badge, not if you display a badge.

In fact, if you score a 0/10 in this check, Scorecard reports "no badge found", but deep down it looks for the form, not the badge. You might not want to display such badge, so it refrain users from completing this check.

If I understand you correctly, that sounds like a bug, let's not enshrine that. I may not be understanding you correctly. Where's the code that does the evaluation?

Here's the current description: https://github.com/ossf/scorecard/blob/main/docs/checks.md#cii-best-practices

The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give full credit to projects that meet the gold criteria, which is a significant achievement for many projects. Lower scores represent a project that is at least working to achieve a badge, with increasingly more points awarded as more criteria are met. gold badge: 10 silver badge: 7 passing badge: 5 in progress badge: 2

I'd like to revisit this grading. "Gold" is really hard to achieve. Earning passing is a big deal. I'd suggest this kind of ranking instead:

• Gold: 10
• Silver: 9
• Passing: 8
• In progress: Rate 1..5 depending on progress (1=started but 0 progress, 5=99%, convert to closest integer).
• Not in progress: 0

[david-a-wheeler]

I think you're using the term "badge" to mean a graphical image, but in the "best practices badge" we mean the English word, that is, something like an indication of meeting certain criteria. It doesn't matter if a graphical image is displayed, the issue is, "has this project met certain criteria"?

[david-a-wheeler]

I think the solution is to change the message "no badge found" to "no effort to earn a best practices badge found". It doesn't matter if a graphical badge image is displayed - what matters is if a project is trying to earn a best practices badge.

[david-a-wheeler]

Also: I would delete the text & list beginning with "To earn the passing badge, the project MUST:". That isn't the full list of criteria; the full list is linked-to above.

[gabibguti]

I think you're using the term "badge" to mean a graphical image, but in the "best practices badge" we mean the English word, that is, something like an indication of meeting certain criteria. It doesn't matter if a graphical image is displayed, the issue is, "has this project met certain criteria"?

Yes, I meant "badge" as a graphical image. I do understand it has a different meaning in English now, thanks for the explanation. Still, I believe "badge" in the GitHub context maps to README badges such as workflow status badges, not "meeting certain criteria".

[gabibguti]

If I understand you correctly, that sounds like a bug, let's not enshrine that. I may not be understanding you correctly. Where's the code that does the evaluation?

https://github.com/ossf/scorecard/blob/2bde7ca25be9bdc798a6e6ca40e192a24058e2b4/checks/evaluation/cii_best_practices.go#L44

[david-a-wheeler]

Yes, I meant "badge" as a graphical image. I do understand it has a different meaning in English now, thanks for the explanation. ...

The project is literally named the "OpenSSF Best Practices badge", and I think no one is interested in a rename.

This is not a big deal, I think the Scorecard text could be tweaked to make this very clear. Here's one proposal.

FIRST: The Scorecard README https://github.com/ossf/scorecard says:

Does the project have an [OpenSSF (formerly CII) Best Practices Badge](https://bestpractices.coreinfrastructure.org/en)?

I would change that to:

Has the project earned a [OpenSSF (formerly CII) Best Practices Badge](https://bestpractices.coreinfrastructure.org/) at the passing, silver, or gold level?

Notice the switch from "have" to "earned" (which is what matters), and the express mention of passing / silver / gold level; both make it clear that this isn't just about an image. Also, I removed "/en" from the URL; the "/en" forces English display. Don't force the language unless you know you want the user to only see the English display. In this case, I think you want the user to see the browser's preferred locale.

SECOND:

In the details at https://github.com/ossf/scorecard/blob/main/docs/checks.md#cii-best-practices

I would change this text:

This check determines whether the project has earned an OpenSSF (formerly CII) Best Practices Badge, which indicates that the project uses a set of security-focused best development practices for open source software. The check uses the URL for the Git repo and the OpenSSF Best Practices badge API.

The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give full credit to projects that meet the [gold criteria](https://bestpractices.coreinfrastructure.org/criteria/2), which is a significant achievement for many projects. Lower scores represent a project that is at least working to achieve a badge, with increasingly more points awarded as more criteria are met.

Into this:

This check determines whether the project has earned an OpenSSF (formerly CII) Best Practices Badge at the passing, silver, or gold level. The OpenSSF Best Practices badge indicates whether or not the project uses a set of security-focused best development practices for open source software. The check uses the URL for the Git repo and the OpenSSF Best Practices badge API.

The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give full credit to projects that meet the [gold criteria](https://bestpractices.coreinfrastructure.org/criteria/2), which is a very significant achievement for projects and requires multiple developers. Lower scores represent a project that has met the silver criteria, passing criteria, or trying achieve the passing badge, is at least working to achieve a badge, with increasingly more points awarded as more criteria are met. Note that even meeting the passing criteria is a significant achievement.

Delete the text "To earn the passing badge, the project MUST:" and the list that follows it. The page already links to the full criteria; listing the criteria here suggests it's the complete list (and it's not).

THIRD:

Regarding:

scorecard/checks/evaluation/cii_best_practices.go

Line 44 in 2bde7ca

Change:

 results = checker.CreateMinScoreResult(name, "no badge detected"

to:

 results = checker.CreateMinScoreResult(name, "no effort to earn a best practices badge found"

FOURTH:

As a separate action, I suggest using a different value ranking (you're renaming it anyway):

• Gold: 10
• Silver: 9
• Passing: 8
• In progress: Rate 1..5 depending on progress (1=started but 0 progress, 5=99%, convert to closest integer).
• Not in progress: 0

Passing is really good, only about 20% of current projects pursuing a badge achieve a passing badge. Gold is really hard; it requires multiple developers, and that fact by itself excludes the majority of OSS projects.

[gabibguti]

I agree with the suggestions

[david-a-wheeler]

Okay, I've create PR https://github.com/ossf/scorecard/pull/2907 to clarify things.

That will give us a better starting point, as it's clearer, but it does NOT resolve this issue. It does not rename the Scorecard criterion, nor does it change the scoring system as suggested above. It simply makes the existing system easier to understand.

So hopefully that PR will be accepted, and that will make it easier to implement this one :-).

[david-a-wheeler]

Okay, back to the main discussion, presuming that PR https://github.com/ossf/scorecard/pull/2907 or something like it will be accepted.

The main problem now is there's a metric named "CII-Best-Practices" that should be renamed to "OpenSSF-Best-Practices". If we're going to rename it, I suggest also updating to a different scoring system, it's an ideal time to do it. My recommendation, as noted above:

• Gold: 10
• Silver: 9
• Passing: 8
• In progress: Rate 1..5 depending on progress (1=started but 0 progress, 5=99%, convert to closest integer).
• Not in progress: 0

I personally think the metric should have more weight in Scorecard, but keeping the existing weight is okay if others prefer it as-is. I think it's more important to have a better scoring regardless of its weight.

If possible, I'd love for the Scorecard JSON file to refer to the URL of the Best Practices badge entry. That way, readers of this metric could quickly jump to the badge entry to learn more information. E.g., here's an example of a current result:

{"name":"CII-Best-Practices","score":10,"reason":"badge detected: gold","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/e1d4f3726920cc49615c1cae8d920a75975500d3/docs/checks.md#cii-best-practices"}}

Somewhere in there it'd be great to have a link to https://bestpractices.coreinfrastructure.org/en/projects/1 (e.g., inside "documentation" add "additional-information": "https://bestpractices.coreinfrastructure.org/en/projects/1"), so that it's easy to start from the Scorecard data and go to the Best Practices Badge data.

I don't know if it's important to continue to support the older metric "CII-Best-Practices". If you do, you could keep the old score with a weight of "0". But unless some user needs it, I'd just drop it, it's confusing & it's not clear it's helpful for backwards compatibility.

[jsoref]

@david-a-wheeler https://github.com/ossf/scorecard/issues/1549#issuecomment-1522153656 is really hard to follow because GitHub forces users to independently horizontally scroll each blob of text.

diff tagged markdown

Here's the laziest improvement to reading the changes (just grouping everything together into ```diff tagged markdown:

-Does the project have an [OpenSSF (formerly CII) Best Practices Badge](https://bestpractices.coreinfrastructure.org/en)?
+Has the project earned a [OpenSSF (formerly CII) Best Practices Badge](https://bestpractices.coreinfrastructure.org/) at the passing, silver, or gold level?

-This check determines whether the project has earned an OpenSSF (formerly CII) Best Practices Badge, which indicates that the project uses a set of security-focused best development practices for open source software. The check uses the URL for the Git repo and the OpenSSF Best Practices badge API.
-
-The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give full credit to projects that meet the [gold criteria](https://bestpractices.coreinfrastructure.org/criteria/2), which is a significant achievement for many projects. Lower scores represent a project that is at least working to achieve a badge, with increasingly more points awarded as more criteria are met.
+This check determines whether the project has earned an OpenSSF (formerly CII) Best Practices Badge at the passing, silver, or gold level. The OpenSSF Best Practices badge indicates whether or not the project uses a set of security-focused best development practices for open source software. The check uses the URL for the Git repo and the OpenSSF Best Practices badge API.
+
+The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give full credit to projects that meet the [gold criteria](https://bestpractices.coreinfrastructure.org/criteria/2), which is a very significant achievement for projects and requires multiple developers. Lower scores represent a project that has met the silver criteria, passing criteria, or trying achieve the passing badge, is at least working to achieve a badge, with increasingly more points awarded as more criteria are met. Note that even meeting the passing criteria is a significant achievement.

- results = checker.CreateMinScoreResult(name, "no badge detected"
+ results = checker.CreateMinScoreResult(name, "no effort to earn a best practices badge found"

diff tagged markdown with text wrapped for readability

-Does the project have an
+Has the project earned a
 [OpenSSF (formerly CII) Best Practices Badge](
-https://bestpractices.coreinfrastructure.org/en
-)?
+https://bestpractices.coreinfrastructure.org/
+) at the passing, silver, or gold level?

 This check determines whether the project has earned an OpenSSF (formerly CII) Best Practices
-Badge, which indicates that
+Badge at the passing, silver, or gold level.
+The OpenSSF Best Practices badge indicates whether or not
 the project uses a set of security-focused best development practices for open source software.
 The check uses the URL for the Git repo and the OpenSSF Best Practices badge API.

 The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold.
 We give full credit to projects that meet the [gold criteria](https://bestpractices.coreinfrastructure.org/criteria/2),
-which is a significant achievement for many projects.
+which is a very significant achievement for projects and requires multiple developers.
 Lower scores represent a project that
-is at least working to achieve a badge,
+has met the silver criteria, passing criteria, or trying achieve the passing badge,
+is at least working to achieve a badge,
 with increasingly more points awarded as more criteria are met.
+Note that even meeting the passing criteria is a significant achievement.

 results = checker.CreateMinScoreResult(name,
-"no badge detected"
+"no effort to earn a best practices badge found"

--- fwiw, the text changes proposed seem quite reasonable (once I'm able to actually compare before/after).

[david-a-wheeler]

jsoref - I think the text changes were already merged in https://github.com/ossf/scorecard/pull/2907 - I think what's undone is renaming the criterion (which changes the name of a key!).

[afmarcum]

Newer structure probe referred to as OpenSSF Best Practices, but older references remain CII Best Practices.

Review old references in code and documentation and update. ~1 hr effort.

[spencerschrock]

but older references remain CII Best Practices

I believe we intentionally didn't rename the check in the code, for both backwards compatibility reasons, as well as BigQuery reasons.