Define and clarify scope of SLSA (compare to CNCF supply chain white paper)

[mlieberman85]

This is a placeholder for initial discussion and work. This came out of 8/11/2021's SLSA community meeting. Currently it is not completely clear what is or is out of scope for SLSA.

Background: The nature of the supply chain starts at hardware and ends with production. This is a very broad scope that at some level includes things like thinking about where to root trust, how to assess dependencies, how to build software securely and how to monitor production for software whose provenance can't be verified. When doing the aforementioned activities you also need to think about securing the software, tools and hardware that are being used in them.

For some folks, like myself, it is difficult to ascertain what activities are within the SLSA framework's scope or if SLSA currently has a defined scope.

[mlieberman85]

I work with @apmarshall on supply chain security at the CNCF wrote up a good article and comparison of the CNCF supply chain security white paper and SLSA and here's the table with the comparison. Disclosure: I was also a contributor to the CNCF supply chain white paper.

I think there are some things that fall under the scope of one of the categories like "securing the source code" that we might want to consider whether or not they're in scope of SLSA. There are also probably some things where it doesn't make sense either. Also I know in some of the other open issues some of these things are already being discussed.

[inferno-chromium]

Thank you @mlieberman85 for sharing your valuable insights and comparison of CNCF white paper with SLSA.

[TomHennen]

@inferno-chromium I think we do want an issue that's just around clarifying the scope of SLSA. I suppose we can create a new one for that purpose?

Thanks for the comparison with CNCF @mlieberman85 and @apmarshall. It was an interesting read. I had a couple of thoughts about the write-up.

From the chart

"Securing the source code"

SLSA probably fails here because it just isn’t opinionated on how the source code should be secured? E.g. SLSA doesn’t require signed commits and doesn’t do full verification of commits to a branch. Instead it delegates a lot of this work/trust to the source control system itself. The source control system might choose to secure itself using some of these methods, but SLSA doesn’t mandate it.

“Cryptographically guarantee policy adherence”

As I read it this is because SLSA doesn’t yet have any requirements about how or when to actually verify these SLSA properties. Is that right?

"Securing Materials"

The first four rows of this section seem to be because SLSA doesn’t currently address the transitivity issue. Is that right? One of the things we’ve been thinking about is that policy could say “software package X must have SLSA level 4 and transitive SLSA level 3” where the transitive SLSA level would apply to all of an artifact’s transitive dependencies. If that approach were taken I think SLSA could cover these cases.

I see we’ve already talked about this in Slack, so perhaps the biggest issue you see is that this isn’t documented anywhere yet or encoded in a level? In some ways perhaps this gets back to the fact that SLSA doesn’t require anyone to check the provenance and has no guidance on what to check.

Is this something we should try to address sooner rather than later? How this could be done/verified is still under development. Would it make any sense to add an issue that addresses things that could be done (like having a policy require a specific transitive SLSA level, or adding a higher SLSA level that includes requirements on transitive dependencies)?

“Validate runtime security of build workers”

Is the main issue that SLSA doesn’t have any specific requirements on how the builder workers should be secured? Would it make sense to say we recommend you “follow the CNCF, X, or Y guidelines to secure your build worker”? Would an alternative way to resolve this be some sort of accreditation process for SLSA. FWIW we've been trying to avoid being too prescriptive where possible because we figure folks would have opinions on the various ways they want to secure their systems. I think there's definitely space to point people at established methods of securing things without mandating them. WDYT?

“Build and related CI/CD steps … delivered as code”

This is actually a bit of a mistake at initial publication, something that it’s been proposed we roll-back at levels 1 & 2. (3 & 4 should definitely maintain this property).

From the article

“But this doesn’t do anything to address the contents of that package: is it malicious?”

This is probably fair. SLSA does prevent people from adding malicious code at higher levels due to requiring 2 party review. But it doesn’t do any independent scanning or analysis of the code to detect issues that might have slipped past review. It also doesn’t necessarily help if you don’t require code review (SLSA levels 1-3) and you don’t think that being discovered as the author of a malicious change has significant deterrent effect. Does that reflect your thinking?

“SLSA doesn’t address securing deployments”

We do have these use cases published, but they’re not required and they’re under-specified at the moment. We'd thought about including a 'policy' component to the SLSA levels but decided it was better to see what the community thought first. Hopefully we'll publish some examples of how we'd like to use policies with SLSA. Depending on what the community thinks there's probably space to actually have some requirements here. WDYT?

[TomHennen]

I wonder if the key takeaways from this analysis are:

1. Should SLSA be more specific about how to do things like secure platforms, secure builders, etc... or should SLSA be less opinionated and direct people to other standards/examples they could use to meet the SLSA requirements?
2. Should SLSA define something about how to apply it to dependencies?
3. Should SLSA define, or least provide examples, for how to secure deployments by actually evaluating the SLSA provenance against policy?

Have I missed anything important @mlieberman85 ?

[mlieberman85]

I think you largely hit everything. I think there's a breadth and depth thing here. Should SLSA be broader than it is, as you mentioned focus on stuff like infra security, secure builders? And in the stuff that SLSA does focus on like provenance should we go deeper like how we apply policy against SLSA to actually base the work on the provenance we're getting.

[apmarshall]

Thanks, @mlieberman85 for copying me into this conversation.

For a little context: I wrote the article because we are working now on a reference architecture for supply chain security based on the CNCF paper. In our conversations, SLSA came up a few times as another possible framework we could take our cues from, and I hadn't read it yet. So I sat down and read through the docs and made the chart to help myself get a handle on the similarities and differences. The big categories and the controls in the left column of the chart are directly from the CNCF paper. I'm not intending to say that they are exactly what SLSA should be requiring, just quoting from the CNCF list. There are several that are not exactly the same but similar across both frameworks, and I tried to note that in the chart.

My big take away, and I would love to hear from this group of people if I'm right or wrong about this, is that SLSA is principally aimed at the question of provenance. I think of provenance as being the most clearly measurable/verifiable aspect of supply chain security, so that seems like a very good focus for a framework that is going to provide anything like a "level" or "score." CNCF was aiming for a much broader set of concerns (and not providing any sort of scoring metric), and I think there is always going to be debate about the edges of that "canon" of best practices. I don't think SLSA needs to adopt everything in the CNCF guide, especially those things that are not particularly relevant to provenance.

I do think there's a few things in the CNCF guide that are provenance relevant and worth considering for SLSA. For example: commit signing provides some provenance about the source of the source code itself. I could see an argument that SLSA is concerned with whether the final artifact reflects the source code that went into the build pipeline and where that source code itself came from is out of scope. But I could also see an argument that says knowing the commits came from who they claim to come from is really relevant to whether the package is trustworthy. So maybe that's a debate worth having. There's a few others (the things I highlighted in yellow, mostly) that I think there could be similar discussions about as SLSA is clarifying its scope.

[inferno-chromium]

Thank you @TomHennen @mlieberman85 @apmarshall for the last set of comments. Do you think we should break this into individual new issues and bring it up in next SLSA sync ?

[trishankatdatadog]

Thank you @TomHennen @mlieberman85 @apmarshall for the last set of comments. Do you think we should break this into individual new issues and bring it up in next SLSA sync ?

I have some comments I haven't pushed yet, sorry...

[inferno-chromium]

Thank you @TomHennen @mlieberman85 @apmarshall for the last set of comments. Do you think we should break this into individual new issues and bring it up in next SLSA sync ?

I have some comments I haven't pushed yet, sorry...

No rush, take your time.

[TomHennen]

SLSA is principally aimed at the question of provenance

I think that's probably fair, even more fair if 'provenance' is interpreted broadly to include things like "how can we trust that what this provenance says is true".

I could see an argument that SLSA is concerned with whether the final artifact reflects the source code that went into the build pipeline and where that source code itself came from is out of scope

Ah, I'm not sure that's true. I think SLSA does care about where the source came from and who it came from. The source requirements do have things like "Each change must contain: the identities of the uploader and reviewers...", "Every change in the revision's history has at least one strongly authenticated actor identity (author, uploader, reviewer, etc.)", and "Every change in the revision's history was agreed to by two trusted persons".

Currently we only have the SLSA provenance which lets builder record the (git repo, branch, commits) that were used in a build and then there's this outstanding issue on how to convey information about how the source control systems themselves were secured. SLSA doesn't currently talk about things like commit signing. In part that's probably because 1) we want different implementations to figure out how they want to meet the requirements (there are probably source control systems that don't support commit signing but might have some other mechanisms that could be used) and 2) we haven't gotten around to it or just don't know what the right set of recommendations are.

But I could also see an argument that says knowing the commits came from who they claim to come from is really relevant to whether the package is trustworthy

Yes, I agree completely!

[mlieberman85]

I think that's probably fair, even more fair if 'provenance' is interpreted broadly to include things like "how can we trust that what this provenance says is true".

I think "how we can trust what this provenance says is true" is huge. What's the bottom turtle as they say. I'm being facetious but do we want to recommend that you need to fabricate your own hardware from scratch? Probably not. Do we want to recommend using zero-trust, minimal os, etc. probably.

[trishankatdatadog]

I really like the framing of provenance, and agree that that is what SLSA should try to better define and provide more measures of. One important problem that SLSA does not yet address is how to authenticate provenances in the first place. I believe the CNCF whitepaper calls this problem "securing deployments".

[TomHennen]

Regarding authenticating provenance, I think a resolution to #101 might help with that?

[trishankatdatadog]

Regarding authenticating provenance, I think a resolution to #101 might help with that?

Oh, yes, makes sense, I forgot about that issue, thanks for pointing out! @joshuagl should we get started?

[dlorenc]

Cc @asraa as well, we were just talking about how the TUF/sigstore integration could help here

[trishankatdatadog]

Cc @asraa as well, we were just talking about how the TUF/sigstore integration could help here

We should co-author a blog post...

[joshuagl]

Count me in!

[bureado]

Now https://docs.google.com/document/d/1FwyOIDramwCnivuvUxrMmHmCr02ARoA3jw76o1mGfGQ/edit and https://github.com/thesecuresoftwarefactory/ssf

[mlieberman85]

Now https://docs.google.com/document/d/1FwyOIDramwCnivuvUxrMmHmCr02ARoA3jw76o1mGfGQ/edit and https://github.com/thesecuresoftwarefactory/ssf

I can definitely help out there. I will say both of those do cite and/or integrate with SLSA directly. The SSF project is intended to be an implementation of the ref arch document as well as a showcase of how to consume SLSA attestations for helping secure the supply chain of the SSF itself while also then letting people build artifacts that themselves have SLSA attestations that you can trust because not only can you establish provenance within the SSF but you can audit the SSF itself for its attestations and do it transitively a couple of levels.