Future Suggestion: Build Systems Track

[mattfarina]

The "build track" appears to look at things from the standpoint of a user of a build system. In the future directions sections it covers expanding the "build track" and a "source track".

While these are great, there is a track that's missing. A build systems track that covers the security of a build system. The material on verifying build systems is a nice start. But, there are some limitations with it:

1. In many cases people aren't empowered to do a self assessment of a build system.
2. Folks can't see, at a glance, how a provider does. We are accustomed to being able to view things at a glance.
3. Without something more formal you can't have a compliance program. Given the attack vectors that can come through a build service, it would be great to have a 3rd party compliance program for them. To do that you need something to comply against. #515 showed me that folks were thinking of compliance.

Note, to expand on why I can't verify a build system here's a simple example. I cannot see who has superadmin access and can make arbitrary changes. Or, how an organization handles that level of access.

A build system could be a complete mess. Very much insecure. Or, it could be really good. At present, there is no way to know and nothing in the future plans to solve for it.

I came to this when I noticed the Common Requirements have been removed. This is already captured in #674.

[TheFoxAtWork]

This is a good call out @mattfarina. Given that a significant portion of open source projects rely heavily on managed build systems, conveying the security of build systems to users is important, akin to how 3P Cloud Providers convey the security of their services to their users. When we (CNCF TAG Security) were developing the Supply Chain Security Whitepaper, this was a point of long discussion that is often overlooked when we talk about the supply chain - the security controls, constraints, and considerations in the design and architecture of a given build system. This is why there is an entire section of the security of a build infrastructure within the paper. Its not perfect, but by approaching it from the perspective owning the breadth of the build system, we were able to identify areas where weakness in the build infrastructure itself could interfere with the integrity of the artifacts being built.

CC: @jonmuk who was instrumental to our development of the paper.

[kpk47]

Thank you for bringing this up! @JoshuaMulliken and I are still working on #515 but got a bit sidetracked working on the v1.0-RC2 spec release. Now that the release is ready, we'll be focusing on the conformance program more.

Do you think there would be value in a Build Systems track as opposed to a yes/no conformance program? Right now we're focused on whether build systems are isolated sufficiently to meet Build L3, but it would make sense to have a higher level of certification for Build L4. I wonder if there would be value in lower levels of a Build Systems track (e.g. if a build system needs to hit Build Systems L3 to produce Build L3 provenance, would there be any value in a Build Systems L1 or L2? What would they look like?).

[mattfarina]

Do you think there would be value in a Build Systems track as opposed to a yes/no conformance program?

In short, yes.

The CNCF TAG Security white paper notes:

Build infrastructure should have the same level of security configuration as its generated artefacts' target runtime environment.

This is something I've learned matters. What one needs for Finance, Healthcare, and Government sectors is often different from a typical small to medium sized business. For example, there are build services that don't allow network/internet access while the build step is happening. In the 0.1 SLSA spec there were requirements for access and superusers. Some build systems will target mass audiences. Others may target niches with stricter requirements. Being able to spot that matters.

I like to think of it this way. All build systems have attack surfaces. How are those managed and are they managed to a level that meets my needs? My needs being different depending on the situation.

While I was reading Verifying build systems I noticed the questions are very open ended. For example,

What are the ways an employee can use privileged access to influence a build or provenance generation? Examples: physical access, terminal access, access to cryptographic secrets

For end users who consume a build system, they aren't looking for a yes/no in general. People have different requirements that can't be answered with a yes/no. But, things can often be put in terms of levels (or something similar).

There is also the user experience. This is for both consumers and producers. Consumers want to see, at a glance, where something is. Producers going after levels is a bit of gamification that can help them make their systems more secure.

Does that make sense?

[mlieberman85]

There's a lot SLSA is working on in this space. We're playing a delicate balance between granularity and standardization. I agree with the Build Systems Track, and previous suggestions have been to look at white papers folks like the CNCF have released, e.g. Secure Software Factory Ref Arch, and Supply Chain Security White Paper, NIST SSDF, etc. and refer to those practices as opposed to creating new requirements.

@mattfarina Would something like an attestation (similar to the provenance spec), that makes claims on the build system might help here? It would be a signed JSON document with a bunch of metadata claims about how the build system is built, secured, and operated? This would allow for both self signed attestations or allow for other external parties make claims via an audit/conformance program.

(FWIW, I wrote up some of the securing the build and build infrastructure pieces of the Supply Chain Security White Paper and helped lead up the CNCF Secure Software Factory)

[mattfarina]

Would something like an attestation (similar to the provenance spec), that makes claims on the build system might help here?

I've been thinking about this for a bit and there are a couple considerations to think about:

1. Who is the target audience of the output? For example, with the SLSA levels it provides consumers with an at a glance view and it gameifies those building things to try to improve their level. It goes beyond attesting to some technical detail. A lot of people need or can otherwise use that. For build systems, how do you convey how good their doing at a glace? attesting to some claims in signed JSON doesn't do that. I'm not saying attesting to claims isn't useful. It just doesn't cover all needed cases.
2. The build levels in RC 2 aren't linked to any level of build systems. I realized that someone could have a poorly operated (from a security perspective) build system and still get to the highest SLSA build level in the new levels. I realize this could still be the case with the 0.1 L3 but the 0.1 L4 put some controls into place that forced some good practices. Those were the common requirements that have been shifted out of the build and source tracks. attestations don't help with this problem. A system can attest to some poor practices.

This is why I don't think attestations are enough.

[arewm]

An attestation does not help to provide evidence of the practices of the build platform. I interpret the challenge here to be around the "trust but verify" statements. Attestations help to associate the platform's sign-off on artifacts. This can be used by consumers once the platform is trusted but there is no way for producers/consumers to verify the trust placed in build platforms.

Many CI systems used today are trusted on first use, but having been in the middle of reacting to an incident in a once-trusted system (i.e. Travis), it is much harder for systems to gain my trust from a producer's point of view. I haven't knowingly been a consumer of CI systems that have had leaks, but that is another element that I see Matt trying to get at (and a much harder problem to solve as I see producer trust as being easier to get than consumer trust).

[MarkLodato]

Thanks, @mattfarina, for joining the SLSA Specification meeting today and leading the discussion! To recap:

• Agreed on the value of covering this.
• Not a blocker for v1.0 since this was only a SLSA 4 requirement of v0.1, which is not defined in v1.0.
• Stop-gap solution is to add this topic to docs/spec/v1.0/future-directions.md.

@mattfarina volunteered to send out a PR by Wednesday (?) to add to Future Directions. Thank you!