Trustworthiness of provenances in Build L2

[behnazh-w]

The trustworthiness of provenances in Build L2 needs clarification. The requirement states that the trustworthiness of the provenance is ensured. However, without hardening the build platform, this requirement will not be possible. In other words, if an attacker can access the signing material by breaking out of the build environment the trustworthiness of provenances cannot be ensured for Build L2.

This problem makes the Forge values of the provenance (other than output digest) threat example hard to understand because without some level of hardening the build platform, I'm not sure how this attack can be prevented at level 2.

[mlieberman85]

So the way I read it, and maybe we can make it clearer here is the build is not the same as the build platform. The build platform should always be secured against tampering from the build itself.

Build L3 focuses on builds tampering with each other.

[houdini91]

I was thinking about this , IMHO untrusted build platform may mean many things .

For example , That may actually prevent runs from influencing one another, even within the same project.

But does not prevent secret material used to sign the provenance from being accessible to the user-defined build steps.

In this case if you trust the developers who have access to a specific project you can still trust the prov was not effected by the other projects users and builds.

You can also further harden access to users using commit signatures restriction users to specific files.

Another approach compare the prov details with the build platform API directly. For example verify the build number and commit and timestamp of the prov match the event seen in the platform API.

Verifing prov against the rekor transparency.

Another thing I was thinking about is basic key management tricks, rolling keys, replacing keys (past and future protection), Signing using Multiauple keys

As well a range of key management tools services that you may want to use instead of the untrusted build platform one..

The same goes for separating between builds, hardware features allowing to separate trusted and untrusted code (vm, SGX, namespaces, using simply seperate build machines.

[MarkLodato]

The requirement states that the trustworthiness of the provenance is ensured. However, without hardening the build platform, this requirement will not be possible.

@behnazh-w I'm not sure how to interpret your comment. Do you mean:

• That ensuring trustworthiness of the provenance only really comes at Build L3, not Build L2? And that Forge values of the provenance should be marked as "Build L3" not "Build L2+"? If so, could you be more specific about what your suggested changes are?

• That the Build Track in general cannot really ensure trustworthiness of the provenance without talking about how the build platform is implemented and operated (#802)?

• Specifically with the following quote?

  This may be the original build, an after-the-fact reproducible build, or some equivalent platform that ensures the trustworthiness of the provenance.

• Something else?

[behnazh-w]

• That ensuring trustworthiness of the provenance only really comes at Build L3, not Build L2? And that Forge values of the provenance should be marked as "Build L3" not "Build L2+"? If so, could you be more specific about what your suggested changes are?

Comparing the requirements in Build L2 and Build L3, L2 asks for trustworthiness of a provenance without giving any details. L3 makes it more concrete by stating:

prevent secret material used to sign the provenance from being accessible to the user-defined build steps.

Now, I would argue that the above property should also be required at Build L2 if we really want to trust the provenance and prevent Forge values of the provenance attacks. So, my suggestion is to add this requirement to Build L2, and come up with a stronger requirement for Build L3, e.g.,

• prevent secret material used to sign the provenance from being accessible to project administrators and maintainers.

This would be achievable in build platforms, such as GitHub Actions when they supports OpenID Connect tokens, and keyless signing as a result.

• That the Build Track in general cannot really ensure trustworthiness of the provenance without talking about how the build platform is implemented and operated (https://github.com/slsa-framework/slsa/issues/802)

To me the specification first needs to clarify the main differences between Build L2 and L3 in terms if trustworthiness of a provenance. Next, having a Build Platform Operations track in future would help to assess whether the requirements are met.

[sudo-bmitch]

Perhaps a way to think of the differences is "who do you need to trust?"

• L1: unsigned provenance requires you trust the entire chain, including the distribution.
• L2: you no longer need to trust the distribution. But you must trust the developer of the CI pipeline.
• L3: you trust the CI platform, but don't need to trust the pipeline or distribution.

[MarkLodato]

I agree with @sudo-bmitch's description. From https://slsa.dev/spec/v1.0/levels#build-l2:

[At Build L2:] Forging the provenance or evading verification requires an explicit “attack”, though this may be easy to perform. Deters unsophisticated adversaries or those who face legal or financial risk.

[MarkLodato]

Discussed at the 2023-08-14 spec meeting. Next steps:

• In the short term, fix the misleading ... ensures the trustworthiness of the provenance language. Originally that was meant as a qualifier on "some equivalent platform" but it may be misinterpreted as applying to the whole bullet. An alternative phrasing could focus on the value over L1, e.g. "resists tampering by the developer".
• Longer term (possibly v1.1+), reframe levels.md around around the idea from @sudo-bmitch: "who do you need to trust?" There was general agreement that this would be valuable. We would probably want to not just add more bullets (each entry is already long!) but instead rework the "Summary", "Intended for", and/or "Benefits" bullets.

[joshuagl]

Thanks for summarising @MarkLodato! I think we should move the (excellent) longer term idea into a separate issue and track only the short term item in this issue.

@behnazh-w, would the proposed short-term fix resolve this issue for you?

[behnazh-w]

The plan to have a short-term and a long-term fix sounds good to me.

I'm not sure though what the short-term fix would look like. @MarkLodato can you please clarify what "resists tampering by the developer" means in terms of "who/what to trust" and how the requirement would look like after the fix?

[MarkLodato]

I'll send out a PR to discuss potential wording.

@joshuagl do you mind filing a separate issue for the long-term fix (presumably for v1.1)?

[MarkLodato]

Sent #948 for review.

[joshuagl]

@joshuagl do you mind filing a separate issue for the long-term fix (presumably for v1.1)?

Done. https://github.com/slsa-framework/slsa/issues/949

[MarkLodato]

Open comment from https://github.com/slsa-framework/slsa/pull/948#discussion_r1357706452 by @behnazh-w:

Do we think we're good to submit this PR as-is, or are further changes needed? If it's strictly better than the old version but still not perfect, my preference would be to just submit it and then have a separate PR to iterate.

We can merge this PR as is and refine the following definitions and requirements in a separate PR:

• The definition of "dedicated infrastructure" at SLSA Build L2
• Change the Accuracy part of Provenance is Authentic requirement or add a new requirement that does not mandate the following for SLSA Build L2, so that npm provenances can satisfy L2.

The provenance MUST be generated by the control plane (i.e. within the trust boundary identified in the provenance) and not by a tenant of the build platform (i.e. outside the trust boundary)

[MarkLodato]

Upon further reflection (background), I think the main issue is that the security objective of Build L2 is too fuzzy. Currently we have:

Requirement	L1	L2	L3
Provenance exists	✓	✓	✓
Provenance authentic		✓	✓
Provenance unforgeable			✓
Builds hosted		✓	✓
Builds isolated			✓

The requirements "authentic" and "hosted" are wishy-washy with no clear security bar. Quote (emphasis added):

[At L2:] The build platform MUST have some security control to prevent tenants from tampering with the provenance. However, there is no minimum bound on the strength.

What if we simplify the ladder by moving "unforgeable" to L2 and drop/merge the redundant requirements? Then we'd have:

Requirement	L1	L2	L3	(alternative wording)
Provenance exists	✓	✓	✓	Provenance existence
Provenance unforgeable		✓	✓	Provenance integrity
Builds isolated			✓	Build integrity

In other words:

• At L1, provenance is simply untrustworthy.
• At L2, provenance itself is trustworthy—it really was generated from the platform and external parameters claimed—but an attacker could have tampered with it during execution. In other words, L2 protects after the build.
• At L3, provenance is trustworthy and the build has protections against tampering. In other words, L3 protects during or after the build.

I think this might make things more clear for everyone. Thoughts?

If we agree that this is indeed better, the question is then, how disruptive would this be? It would definitely require a v2.0 since it's not backwards compatible. But I'm more wondering how many systems would this change affect, i.e. how many have already implemented something that passes v1.0's Build L2 but would not pass v2.0's?

[jkjell]

My experiences with build platforms would bias me toward Builds isolated at level 2 and Provenance unforgeable at level 3. The build platforms I have experience with, Concourse, Tekton, kpack (buildpacks), and I think (hope 🤞) all SaaS solutions (Google Cloud Build, Github Actions, Gitlab Pipelines, etc) offer isolated builds. It has felt like (I could be wrong) that Provenance Unforgeable has been harder to achieve. Separating out the provenance generation to a control plane requires changes to those build platforms (it's often something a consumer can't [easily] do).

[mlieberman85]

I think Provenance Unforgeable can still be met with just having non-end user controlled build steps. Trusted steps that don't run arbitrary things that record evidence from the end user step should be enough.

[MarkLodato]

@jkjell That's a good point. However, I think Provenance unforgeable has significant value without Builds isolated, while the opposite is not really true in the context of the Build track. The thrust of the track is having trustworthy provenance:

• Isolating builds without unforgeable provenance does little to make the provenance more trustworthy. Anyone could have faked it.
• Unforgeable provenance without isolated builds at least provides guarantees about how the build was initiated. It protects against attacks after the build, i.e. anyone who does not have access to the build itself.

I think the big question is, are there real-world build platforms that would meet Provenance unforgeable without Builds isolated? This needs a closer look at the definition of Builds isolated. I imagine there are many systems out that lack strong isolation within a tenants, e.g. one tenant can initiate two builds in the same project and have one influence the other because they're in the same security domain. But to know for sure we need to both better define the isolation requirements and map them to real-world systems.

What do you think?

[jkjell]

Yeah, that makes sense. I think I often conflate build and control plane isolation. Then I make the leap to the idea that if builds are not isolate, the control plane may not be either (viewing isolation as the prerequisite to each). I think many times the mechanism for this isolation (say containers or clusters per boundary) can be the same for each but, that's not generally true. I'm not sure that Provenance Unforgeable is a directly translation to that but, it's where my brain wants to go. 😅

This is a long way to saying ➕ to better defining isolation requirements and possibly considering its relationship to build system control planes while doing so.

[behnazh-w]

What if we simplify the ladder by moving "unforgeable" to L2 and drop/merge the redundant requirements?

I think its' a good idea to be more specific about unforgeable provenances:

• forging provenances during the build, which requires running malicious code during the build
• forging provenances after the build, which requires breaking the digital signature

Requirement	L1	L2	L3	(alternative wording)
Provenance exists	✓	✓	✓	Provenance existence
Provenance unforgeable after the build		✓	✓	Provenance integrity
Provenance unforgeable during the build (isolated)			✓	Provenance integrity
Builds isolated			✓	Build integrity

I would also keep Builds isolated separate from Provenance isolated because it is possible to have an isolated provenance generation while the build is not isolated (e.g., generic provenances that use GitHub Actions Reusable workflow produce provenances in an isolated VM but the build itself can run in the VM where untrusted code runs).

[joshuagl]

What if we simplify the ladder by moving "unforgeable" to L2 and drop/merge the redundant requirements?

I'm all for moving unforgeable to L2 and merging with authentic.

The hosted requirement causes a lot of confusion and I think it makes sense to remove it from L2. What happens to it then, though? I think we can all agree that we should be discouraging software producers from generating release products on users' workstations, for security and business continuity reasons. Perhaps we can make this more explicit in verifiying systems until such a time as we have a concrete control and outcome to document in the Build track?

In other words:

* At L1, provenance is simply untrustworthy.

* At L2, provenance itself is trustworthy—it really was generated from the platform and external parameters claimed—but an attacker could have tampered with it during execution. In other words, L2 protects _after the build_.

* At L3, provenance is trustworthy _and_ the build has protections against tampering. In other words, L3 protects _during or after the build_.

The stated outcomes here reflect the focus areas we document on the levels page of the v1.0 spec:

how disruptive would this be? It would definitely require a v2.0 since it's not backwards compatible. But I'm more wondering how many systems would this change affect, i.e. how many have already implemented something that passes v1.0's Build L2 but would not pass v2.0's?

Tekton chains and npm provenance come to mind as two solutions which claim SLSA Build L2. I believe if we merge provenance authenticated & provenance unforgeable that npm packages using the npm GitHub Action will still meet SLSA Build L2 (thanks to use of per-job workflow identities and Sigstore). I'm much less familiar with Tekton & Chains, but I think we have members of the community who can help us understand whether this change to the spec would be disruptive to those projects.

[sudo-bmitch]

The hosted requirement causes a lot of confusion and I think it makes sense to remove it from L2. What happens to it then, though? I think we can all agree that we should be discouraging software producers from generating release products on users' workstations, for security and business continuity reasons.

I've seen a lot of that confusion. My own take is we should focus on the goal rather than a proxy for that goal. I believe we want to be sure the build server is relatively secure and maintained to reduce the likelihood of a compromised build host.

Otherwise OSS purist that refuse to use a cloud service and instead run a reproducible build on an airgapped laptop in their basement that otherwise stays locked in a cabinet will feel excluded by a technicality. While the insert name of compromised cloud build service users with compromised credentials, or even someone with an unpatched Jenkins server listening on the internet, can keep checking the box.

Perhaps phrasing it as "builds run on a well maintained and secured platform" with references to the CIS benchmarks, NIST, NSA, etc for the process to secure those hosts.

[joshuagl]

The discussion of removing "Builds Hosted" from Build L2 may need to be forked into a separate issue. We keep coming back to this, so I think it's worth some attention. Can we articulate the concrete security outcome we want to achieve through a hosted builds, or similar, requirement?

These don't feel very concrete, but some straw-person proposals:

• Reduced access to the system (i.e., rare, logged, gated as in v0.1's access)
• Reduced likelihood of it being used as an attack vector through general purpose operation (email, less strict controls for installing software, differing workloads, etc)

[arewm]

Tekton chains and npm provenance come to mind as two solutions which claim SLSA Build L2. I believe if we merge provenance authenticated & provenance unforgeable that npm packages using the npm GitHub Action will still meet SLSA Build L2 (thanks to use of per-job workflow identities and Sigstore). I'm much less familiar with Tekton & Chains, but I think we have members of the community who can help us understand whether this change to the spec would be disruptive to those projects.

While I am not a member of the Tekton/Chains community, based on my experience with them, it is possible to leverage them to achieve SLSA 1.0 Build L3+. Some of the challenges come down to ensuring that the ServiceAccount running the pipeline is not over-provisioned and to ensure that data transferred between the build Tasks and steps is properly protected from tampering.

[MarkLodato]

To consider this issue resolved, it should be clear to most readers how to answer #1002 and the related Slack thread.

In particular:

• Call out the two main designs:
  • Traditional: control plane generates and signs the provenance directly, including the output hash.
  • Sigstore: control plane generates and signs certificate containing the provenance (except the output hash), gives it to the workload, and the workload signs an attestation containing the output hash.
• Document the threat in the Sigstore model that the certificate is leaked, and what level (if any) this threat must be addressed at.
  • On the one hand, the only impact is that the attacker can "Forge output digest of the provenance", which we already say is out of scope. So maybe it's another form of the same threat. (If so, we should document that clearly.)
  • On the other hand, this attack is an infoleak (extacting the token) rather than a full-blown remote code execution, and once you extract you have a time window where you can output multiple things with that stolen cred, so overall it's a lower bar for the attacker. So maybe it should be in scope for L3? I dunno, I'm on the fence.
  • Either way, we probably want to differentiate between a "vulnerability" (unintended by the author of the build workload) from "intentional".

In other words, the goal of the attacker is to say that output X was produced from a build of git repo/commit Y, which would not normally produce X. In this new threat, the attacker extracts the OIDC/x509 from a legitimate build of repo/commit Y and sign an attestation saying the output was X (which is false). But if the attacker had enough access to extract the OIDC/x509, couldn't they just have had the original build output X directly? Is that a substantially easier attack? I could go either way.