slsa-framework / slsa

Supply-chain Levels for Software Artifacts
https://slsa.dev
Other
1.56k stars 226 forks source link

Summarized verification results in VSA, timeless vs. time-sensitive #1207

Open AdamZWu opened 1 month ago

AdamZWu commented 1 month ago

VSA is intended to present a summary of verification results, so that consumers (who trust the attestor) can avoid re-perform verification.

However, practically there are two types of verification results:

Is there any guideline / recommendation from SLSA on how to handle the time-sensitive verification results? E.g.:

TomHennen commented 1 month ago

My recommendation is that the VSA can be used to store these time-sensitive results and that consumers that care about timeliness should consult the VSA's https://slsa.dev/spec/v1.0/verification_summary#timeVerified to determine if they're acceptable. E.g. the VSA results won't expire, but the consumers may have a requirement for newer VSAs.

It's likely desirable for VSA producers to indicate somehow, either in documentation or in the property name, just how recent the data used to assert that property is.

E.g.

VSA issuer: "When we set VULN_SCANNED in verifiedLevels that means there was a vulnerability scan no older than 24 hours from the time encoded in timeVerified."

or

VSA issuer: "When we set VULN_SCANNED_24H in verifiedLevels that means there was a vulnerability scan no older than 24 hours from the time encoded in timeVerified."

Note that SLSA itself doesn't currently have any time-sensitive results, but does allow VSA producers to set fields in VSAs if they wish. So, SLSA itself leaves this undefined and you really just have my advice. :)

zachariahcox commented 1 month ago

I agree with @TomHennen 's take.

I hadn't thought about this scenario much before, but it's pretty important to talk about when a VSA (and maybe which version of the VSA) produces the attestation. I suspect it will often be the case that the consumer literally cannot reproduce policy or logic evaluated by the VSA issuer.

This could be either because they don't have access to the tooling or they don't know exactly what the VSA does to determine correctness.

mlieberman85 commented 1 month ago

I agree with @TomHennen as well. I think there's a few subtleties around dynamic or temporal data. I'll throw out a few examples:

It feels like there's a spectrum here of data that mostly stays static to stuff that is very much point in time.


Some food for thought, but slightly tangential: I have found it can be difficult if you have a hierarchy of VSAs. For example:

Which then get turned into a Final Summary VSA that is a VSA that includes summary data on the source and build. If you have something that is time sensitive some of the time sensitive data in Source and Build might get lost. For example the policies. One of the thing that's not super clear to me is what happens if Source and Build VSA requires the attestations it bases its summary to be let's say no older than 7 days and then the Final Summary VSA requires its data to be no older than 7 days you potentially have a situation where the VSAs are less than 7 days old but the data they refer to are themselves older than 7 days. There's ways to work around this but I think it needs to be part of the conversation on how we should use and treat VSAs.

trishankatdatadog commented 1 month ago

Interesting discussion!

@mlieberman85 WDYM by "Source and Build VSA" vs "Final Summary VSA"?

My current thinking on the subject is as follows:

Then, it is up to clients to determine how deep down the rabbit hole they want to go. They can check the VSA timestamps alone (along with the timestamped signatures), or also any "downstream" attestations.

mlieberman85 commented 1 month ago

So, in recent discussions we talked about how most likely you would have different VSAs for different parts of the lifecycle to summarize different things. So as source goes from the source to the build if you have lots of attestations related to the source you would want a VSA for all the things related to source. Then you might want all the things related to build when you go to publish the package.

trishankatdatadog commented 1 month ago

So, in recent discussions we talked about how most likely you would have different VSAs for different parts of the lifecycle to summarize different things. So as source goes from the source to the build if you have lots of attestations related to the source you would want a VSA for all the things related to source. Then you might want all the things related to build when you go to publish the package.

Interesting: so like a VSA per Track, for example? We call this idea partial verification.

AdamZWu commented 1 month ago

How about let's establish two separate concepts:

The time of attestation allows the consumers of attestation to make a subjective decision on whether the attributes being attested fits their ideal time windows (note that they can have more than one time window, such as a tighter window for "best timeliness", a regular window for "acceptable but not ideal", and a wider window of "absolutely must fall within", etc., and make their consumption verification decisions accordingly.)

The time of expiry, however, makes the consumers aware of an objective (to the consumers) advisory of "best before" time advised by the attestor. The consumers mostly likely should handle attestations by taking into account both their own preference above, and the advised deadline here (if present). They could decide to ignore the "best before" advisory -- but some consumers' decision to ignore should not invalidate the need of having this timestamp.

trishankatdatadog commented 1 month ago

How about let's establish two separate concepts:

  • The time of attestation (required)
  • The time of expiry (optional)

You would need a trusted time of attestation (e.g., with RFC 3161) in order to better trust any time of expiry (otherwise anyone with a compromised key can make it up).