Should the source track treat open and closed source projects differently?

[kpk47]

I'm starting to think that the higher levels of the source track will look fundamentally different for closed source and open source projects, and we would benefit from making the distinction explicit in the specification. I'm not entirely sure what that distinction should look like, but one possibility is describing a high level objective in the levels specification and then having open vs closed source implementation guides. For example

Attribute: Available for forensic analysis
Open source version: Source is licensed under a well-known open source license that allows users to copy and retain it for forensic purposes.
Closed source version: Source and associated metadata will be retained for the project's lifetime.

I'm assuming that closed-source projects happen in the context of an organization that can make forward-looking statements and follow through on them. Some open source projects also take place in such contexts (e.g. OSS projects backed by universities, governments, foundations, or corporations), so maybe we'd need a way to account for OSS projects that can also make forward-looking statements. I can imagine a few ways to do that (separate ladders based on org size, higher levels that certain orgs cannot reach), but I want to see if the idea resonates at all before going too deep. Thoughts?

[MarkLodato]

I also think this might be a direction we need to go, but I don't have any thoughts beyond what @kpk47 already said.

[kpk47]

Another possibility comes to mind: we could have a split somewhere based on whether you can make statements about the process used to generate the source or you can only make statements about the source's contents. For example, two-party review, retention and obliteration policies, etc are all about source production process. Static analysis, fuzzing, testing, a blanket "I trust up to this commit" are all about the source's contents (at least in relation to some reader/consumer).

Perhaps closed source users use process-based strategies, and open source users use a content-based strategy. I'm not entirely clear if that would mean different verification flows or a more fundamental change. Alternatively, perhaps content-based strategies are actually the Dependencies Track we've discussed but haven't defined.

[jkjell]

I've been thinking about this from the verification side and parallels to SLSA Build Provenance. The docs for build provenance today separate verifying artifacts and verifying build platforms. Is there a separation similar to that for the source track? A set of requirements verified by a source forge vs a specific repository?

Leveraging a VSA is another possibility for closed source projects. The organization creating closed-source software can still implement all of the checks of all levels of the source track internally. I'm not sure what it would mean to try and implement the source track for closed source software as a consumer.

[kpk47]

I've been thinking about this from the verification side and parallels to SLSA Build Provenance. The docs for build provenance today separate verifying artifacts and verifying build platforms. Is there a separation similar to that for the source track? A set of requirements verified by a source forge vs a specific repository?

Perhaps. It's true that source control systems will need certain features to support the source track, but I haven't been too concerned with how you would verify those features. Source control systems are relatively simple in comparison to build platforms (which have to run user-provided code) and source code is human-readable (whereas built binaries are not), so I don't think we need a platform verification guide. I don't hold that opinion strongly and welcome feedback.

Leveraging a VSA is another possibility for closed source projects. The organization creating closed-source software can still implement all of the checks of all levels of the source track internally. I'm not sure what it would mean to try and implement the source track for closed source software as a consumer.

I don't think a VSA fits here since we're discussing properties of a range of commits (as opposed to a single subject) and the policy is well-defined in the levels specification. Let's focus on what metadata we need to record and who needs to do so. Once we know that, we can figure out what attestation(s) are appropriate.

I didn't mean to suggest that consumers should be implementing the source track for closed-source projects. If you can't inspect the source, then you can't attest to its contents. However, I suspect that there will be a high overlap between projects that are unwilling/unable to meet higher source levels and projects where consumers can inspect the source code and its change history. We should be thinking about what SLSA can say about a project if you have the code (content) but the project isn't using two-party review (process). (Edit: And that might be "use OpenSSF Scorecards," maybe with additional guidance.)

[MarkLodato]

I think the question is less about open vs closed source, and more about the use cases of first-party, OSS third-party, and closed-source third-party (aka "vendor"). For example:

• For first-party, where the software publisher is not trying to convince anyone else, everything is entirely within their control. They can map accounts to humans, set up mandatory 2FA, set up retention for source code repositories, establish standards for code review, and so on. Their entire goal is to reduce risk according to their own calculations. (Reminder: 1P is not necessarily closed source, e.g. it could be an open-source org like Kubernetes protecting itself.)

• For third-party, a consumer (organization or project) is trying to reduce its own risk stemming from the 3P software it consumes. The 1P requirements above are very challenging to apply. We can break this down further into two sub-categories:

  • 3P OSS, where the publisher has absolutely no warranty or contract with the consumer. Most of the requirements are going to be difficult to impose on an open-source project and hard for the consumer to verify. The one that is in the consumer's control is retention, where the consumer can retain a copy. But now it seems like we're almost talking about a separate track. Maybe instead of Source track for OSS, we talk more about a Dependency track?

  • 3P vendor, where the publisher and consumer have some sort of contractual relationship. This again looks different. The publisher can claim that they're doing SLSA and try to convince the consumer to trust them, e.g. through audits or documentation. Further, a contract could strengthen the confidence by the consumer. VSAs might play a role, in that it is an explicit claim of conformance to the SLSA spec.

[jkjell]

@MarkLodato that's great framing and is really helpful for my understanding.

I don't think a VSA fits here since we're discussing properties of a range of commits (as opposed to a single subject) and the policy is well-defined in the levels specification.

Is the subject a commit or a repository? I was thinking to actually validate this policy against a repo, we would need an attestation (or other verifiable, trusted data) for each commit. The policy would then sequentially validate against all commits, reducing the result to the lowest common denominator of any individual commit.

Let's focus on what metadata we need to record and who needs to do so.

Yeah, I'm probably too excited about an end state instead of how we get there. This is reasonable and a requirement for determining how to apply policy. ☺️