Add new checks to support the 2023 Security Slam

[eddie-knight]

Considering the great statistics we see when comparing Slam participants to the greater ecosystem over the past year, the Slam organizers solicited input from a temporary committee to determine goals for this year's effort.

The goals created by that committee include the same metric from last year (CLOMonitor Security to 100%) and 4 more goals for projects to optionally pursue in the upcoming Slam event (which begins on 10/10/2023).

To ensure that the new goals can be at least partially evaluated through CLOMonitor, we collaborated with the OpenSSF Security Insights Specification to incorporate several changes into the spec's latest major release.

With these changes, we will be able to use the Security Insights Specification v1.0.0 to automate a fair portion of the new metrics.

---

Here are all 5 metrics, with added notes regarding how the SI spec will be helpful with each (the last two are just for reference):

1. Complete a TAG Security formatted Security Self-Assessment for each major project element (or repo)
   • security-artifacts.self-assessment allows for an evidence URL that can be evaluated if provided
2. Document software supply chain security decisions for end users
   • dependencies.env-dependencies-policy allows for a URL to the project's dependency consumption/update policy
3. Ensure that every release has an automated mechanism to supply SBOM and provenance artifacts
   • dependencies.sbom.sbom-creation allows users to log a process explanation that can be reviewed manually
4. Ensure each project repo is accounted for within CLOMonitor, Ensure proper check set is assigned to each project repo, Bring security score to 100% for the project
   • n/a
5. Bring all CLOMonitor non-security scores to 100% for the project
   • n/a

Each of the first three metrics above could become a new check under the Documentation category of CLOMonitor checks, and would be executed only for code repos. The checks would confirm the presence of the SECURITY-INSIGHTS.yml file, then look for valid contents in the specified values.

---

This issue is being logged at a short notice due to the need for extensive collaboration with the Security Insights community to ensure compatibility with the Slam goals— we only finally got the necessary changes implemented in today's release.

Because of this, I have blocked off time this week to contribute to CLOMonitor to help incorporate the necessary changes if it's something the project maintainers are happy with adding.

Please let me know here or reach out to me on the CNCF Slack to discuss this more.

[tegioz]

Thanks @eddie-knight!

We could translate this into the following new security checks:

• OpenSSF Security Insights manifest. The project provides a SECURITY-INSIGHTS.yml file that follows the OpenSSF Security Insights specification.

  • This check could help bring visibility into Security Insights project and, even if the parts required for the other new checks to pass are not present, just having it makes projects be a step closer to adding some more security relevant information eventually. So IMHO we could consider its presence as a security best practice. In addition to that, this file could be of great help as it can provide additional places to look for information for other CLOMonitor's checks.

• Security Self-Assessment. The project provides a TAG Security formatted Security Self-Assessment (based on the security insights manifest).

• Dependencies policy. The project provides a dependencies policy that describes how dependencies are consumed and updated (based on the security insights manifest).

Regarding a new SBOM check: given that CLOMonitor already has a very similar check, instead of creating a new one, we could enrich the existing one and make it possible to pass it with the information available in the security insights manifest. It would be like an alternative, standardized way for projects to pass this check. And we could highlight how the SBOM is created when this information is available in the manifest. This would be an example of what I mentioned before about being able to use the security insights information to improve and enrich existing checks.

If this sounds good I think we can have it ready by the end of the week 🙂

[eddie-knight]

Yes, this sounds fantastic- thanks @tegioz!

[tegioz]

I think this is ready @eddie-knight 🙂

As a consequence of adding the new checks, all projects security scores will get a bit worse over the next day (as repositories are processed again).

(screenshot taken from Artifact Hub)

[eddie-knight]

@tegioz This is amazing!

How do you feel about moving some of this to the "Documentation" section, considering that they are indirect rather than direct security improvements?

[tegioz]

Awesome, glad you like it! 🙂

Regarding moving some of them to the documentation section: I'd rather to keep them in security for now and have all the related checks in the same section (like the security policy one, which is similar to the new ones and was already in that section).