search

slsa-framework / slsa

Supply-chain Levels for Software Artifacts
https://slsa.dev
Other
1.52k stars 220 forks source link

Project Roadmap, May 2022 #384

Closed MarkLodato closed 2 years ago

MarkLodato commented 2 years ago

This issue tracks acceptance of Proposal 2: Project roadmap, May 2022.

If you have any questions or comments on the plan, feel free to add them here and I can respond or update the roadmap. Pull requests also welcome.

I suggest that we accept the roadmap in about a week if there are no significant concerns.

joshuagl commented 2 years ago

Pinging @slsa-framework/slsa-steering-committee to review the proposed project roadmap and raise discussion items here.

david-a-wheeler commented 2 years ago

Pushing towards a "1.0" release is in general the right move. I have a few thoughts:

kimsterv commented 2 years ago

This looks great to me! I can think of more stuff we'd want to add but without owners I think it get noisy fast. If I had to pick one thing to add it would be something around certification.

tracymiranda commented 2 years ago

Great to have a roadmap! +1 for the idea of a community blessed/peer-reviewed specification. Also would be great to see some form of badging program or way oss projects can highlight their slsa status.

MarkLodato commented 2 years ago

@david-a-wheeler, to be clear, the roadmap is just trying to give an flavor of what we're thinking, not a full proposal itself. I'm working on a proposal now, which will need review and approval.

  • "the controversial two-party review requirement" - I'm sad about this one. This would be crazy at SLSA 1 & 2, but it's currently at 4. Would it make sense to reduce to "at least 50% two-party review" at SLSA 4? That's what the OpenSSF Best Practices gold requirement does (though its requirements aren't identical). The idea is to reduce risk (even reviews don't guarantee noticing a problem).
  • "Pare SLSA v1.0 down to only cover build integrity, which is SLSA's core strength." - maybe I missed it, but which requirements would be dropped? Which kept?

I'm planning to propose removing "Source requirements" and "Common requirements" entirely. Instead:

(Disclaimer: I am aggregating and publishing these ideas from various sources. I don't claim full credit.)

melba-lopez commented 2 years ago

Hi @MarkLodato .. speaking with @lumjjb today and he mentioned a new WG spinning up for this. Would love to collaborate on the definitions. Do you have meetings setup yet for this?

As we are suggesting SLSA to enable folks on their supply chain security journey, there are concerns with removing/condensing some of these requirements; especially when they map to SSDF/WHEO. We know companies (in US) will have to comply with WHEO so if we can keep some of these requirements (which also makes sense for SLSA) it would be a win-win for SLSA + adopters.

Additionally, I have a supply chain security framework (end to end) that I shared at OSSNA last week. Would <3 to map this to the old vs new SLSA requirements to also help with visualization of how far the SLSA levels cover organizations.

seaylg commented 2 years ago

Hi @MarkLodato, Thank you for sharing your thoughts. I agree with @melba-lopez and also wanted to add the following concerns with removing source integrity and common security:

In my opinion, SLSA has potential to be that guiding light to help the Open Source Community hone in on what is most important to mitigate threats and not just another checklist that links to another checklist/framework. I would be in favor of removing Common Security as long as it is replaced with more specific supply chain security requirements that help companies meet the SSDF/WHEO as Melba referenced above and NIST SP 800-161r1.

mlieberman85 commented 2 years ago

@seaylg @melba-lopez you both bring up good points that I think we should address.

To give you some context based on some of the conversations we've had thus far, we've been trying to strike a balance for v1.0. We have been trying to focus on requirements and specification for the big gaps. For example, the big gap SLSA was trying to fill first was "provenance." When it comes to supply chain security there's a lot of existing tooling, frameworks, standards, etc. around SCA, build security and similar things already exist. Between SSDF, CNCF Best Practices guide (cited by the SSDF), OWASP SCVS, and others there's a lot of work on this. I think SLSA for now at least should cite and partner with these other standards and frameworks to help provide that bigger end to end supply chain picture at least for now.

I think a lot of the issues listed are good issue, I just think we're for now trying to focus on the provenance step which helps address these issues in another way. For example two person code review is useful, however if you don't have provenance for the code, you don't know if you're actually building against the two person code reviewed code. Separately the two person code review stuff is a huge issue from a tooling perspective.

Another problem is since supply chain security is ostensibly just security of the SDLC, therefore meaning supply chain security is all security that affects the consumption or production of software. At some level this includes most of cybersecurity.

Sorry for the long rant. the tl;dr I think is these are all good issues, some of us are worried that we end up having to spend a ton of time and effort on building consensus around a holistic supply chain security framework right out of the gate instead of focusing on what a lot of us consider the big gap that filling would provide huge return on investment which is around establishing and provenance tracking chain of custody. It sounds like a good solution for now is to cite other standards and frameworks that are already hitting some of this while acknowledging there is more work to be done and SLSA's scope might expand post 1.0.

I do think once we get a 1.0 workshop or working session going we should make sure we address all the concerns.

melba-lopez commented 2 years ago

Thanks for that history @mlieberman85 ! Fully get where you are coming from. Definitely want to make it easy for people to consume. I'm going to try to map SLSA to this framework I've created (see attached). This will be for another time, but the goal is to make sure we show people what's covered and whats not.

Hypothetical question: Can a person just achieve SLSA4 with a ton of vulnerabilities in it? If there is no requirement to ensure scanning or having security policies requirement baked into the build system (i.e. stop the build if high/critical vulns or really old vulns found), then SLSA 4 would not provide organizations with the comfort that they are searching for in SLSA.

I don't know the history here...but if this hasn't been tried .. could we instead get Level 1 and Level 2 requirements "stable" and fully defined and fully attainable? And if then try SLSA3 ... and maybe once we get to SLSA 3 we realize maybe we need a SLSA 5 because the jump is too big from 3 to 4. (Just spitballing)

image
mlieberman85 commented 2 years ago

This is awesome stuff and I look forward to watching your talk once OSS publishes it to youtube. This is a really good diagram that aligns with my mental model at least.

To answer your hypothetical (to be clear this is just my opinion): SLSA helps answer questions like "did I perform SCA on what I built?" SLSA is useful in helping you built out that chain of custody from code to artifact, but should be used in conjunction with other security practices. For example, many supply chain attacks wouldn't catch some normal scans if what ends up getting deployed is not what was scanned. I can't speak for everyone, but over time I would love to see SLSA evolve to become more of that end to end solution.

For your second part, I think you might be on the right track there. I do think the community should sit down and hash out what we're trying to accomplish for v1. My worry which is shared with some others in the community is that we either take on too much too soon and end up taking months or years to build out consensus OR we play it too fast and loose and lead to a ton of confusion on what SLSA actually does for us.

JanZerebecki commented 2 years ago

Replace common requirements with a requirement that the build service provides an explanation

I'd like SLSA to suggest that this be implemented via verified reproducible builds. Which would prevent the build from being compromised unless the builder and most of the verifiers are compromised in the same way.

Generally for SLSA 1.0 I'd really like for it to at the highest level to only consist of requirements that are independently end-to-end machine verified. @MarkLodato Is that acceptable to you?

Source requirements ensure source integrity and I'm not sure how we can ensure build integrity without some level of due diligence at the source and still meet the objective of safeguarding the software supply chain.

While true, ensuring build integrity and code review are not strictly temporally nor causally ordered. In fact most people build first in CI and only then do code review. So ordering these into one level scale might hide progress.

While it is a good idea to have code integrity in the form of git commit signing in place from the beginning, it is entirely possible to take code without any integrity nor provenance and do code review on it in an entirely different project. E.g. there are still projects that only publish a tar without signature nor history. You could then use https://github.com/crev-dev/crev to review that. For the currently level of security that the highest SLSA level targets, that should still be equivalent.

The downside of this flexibility and allowing easy step by step progress is complexity. But people applying SLSA can skip most of it.

Hypothetical question: Can a person just achieve SLSA4 with a ton of vulnerabilities in it? If there is no requirement to ensure scanning or having security policies requirement baked into the build system (i.e. stop the build if high/critical vulns or really old vulns found), then SLSA 4 would not provide organizations with the comfort that they are searching for in SLSA.

Maybe we should rename this one "SLSA build integrity" and later tie that, "SLSA source integrity", "SLSA code review", etc. all together in an "SLSA summary"?

I don't know the history here...but if this hasn't been tried .. could we instead get Level 1 and Level 2 requirements "stable" and fully defined and fully attainable? And if then try SLSA3 ... and maybe once we get to SLSA 3 we realize maybe we need a SLSA 5 because the jump is too big from 3 to 4. (Just spitballing)

The way I imagine it, it may make sense during application of SLSA to skip 1-4 and go straight to level 5 as it is less work. Thus it is important that the highest level properly exists, as to not provide an incentive to disregard SLSA or even worse SLSA requiring lower security mechanisms.

MarkLodato commented 2 years ago

Thanks, all! For this issue, are there any concerns with the roadmap itself? If so, could you suggest specific changes? For example, if the current language makes it sound like removing source requirements has already been decided, should we change that to make it more clear that that is simply something being considered?

Responding to specific comments:

speaking with @lumjjb today and he mentioned a new WG spinning up for this. Would love to collaborate on the definitions. Do you have meetings setup yet for this?

We will be forming the WGs ("work streams") at tomorrow's community meeting, July 7. The v1.0 specification will be the top priority of the "Specification" work stream. I'm looking forward to working more closely together!

[... all the points made above by @melba-lopez @seaylg @JanZerebecki @mlieberman85 ...]

These are really great points. They highlight how this is not an easy problem - there is no obvious best solution. As a next step, I suggest that we jointly write a proposal outlining the problem, the various considerations, and possible solutions. Having these all written in a coherent doc will make it easier for us to form consensus. I've started a skeleton which I'll share with the work stream members.

Replace common requirements with a requirement that the build service provides an explanation

I'd like SLSA to suggest that this be implemented via verified reproducible builds. Which would prevent the build from being compromised unless the builder and most of the verifiers are compromised in the same way.

Generally for SLSA 1.0 I'd really like for it to at the highest level to only consist of requirements that are independently end-to-end machine verified. @MarkLodato Is that acceptable to you?

Let's discuss it in the work stream. I do think verified reproducible builds will play a part, particularly for open source, but I have some concerns around how it would work in practice.

melba-lopez commented 2 years ago

Thanks for that link @MarkLodato I was looking for the roadmap and could not find it. I will look through it. The only thing I did just notice is that we are already past mid 2022, so not sure if that changes the priority levels.

JanZerebecki commented 2 years ago

Let's discuss it in the work stream.

Seems like the list of work streams is here: https://docs.google.com/document/d/1L1gEJMBIvE0IbpFi23FOUByDYlItSYPPJmKdhvJQYsg/edit

And the relevant stream seems to point for the discussion to:

#slsa-specification / https://groups.google.com/g/slsa-specification

I'm not in Slack (haven't received a reply to that ticket, yet), but joined the mailing list.

I do think verified reproducible builds will play a part, particularly for open source, but I have some concerns around how it would work in practice.

Please explain your concerns, so I can respond. It is probably better to do that in the relevant ticket https://github.com/slsa-framework/slsa/issues/5 , but I guess we can cross-link tickets and mailing list threads if you prefer that.

I feel like discussing one specific part might make the road to 1.0 clearer.

MarkLodato commented 2 years ago

Hey folks, it seems like the general consensus is to accept, so I https://github.com/slsa-framework/slsa-proposals/pull/5 to address a few items and mark as accepted. If someone wouldn't mind taking a look and approving, I'd appreciate it.