Attestation for policy verification

[kpk47]

SLSA has the concept of a Verification Summary Attestation (VSA). VSA is an in-toto attestation that indicates the attestor verified an artifact's SLSA metadata and found it to meet a certain SLSA level. SLSA users can use VSAs to cache verification decisions and/or delegate such decisions to another trusted party. VSAs are specific to SLSA, but perhaps it would be useful to have an attestation that was agnostic to the type of policy the attestor used. That attestation would mean “ evaluated against and it (passed|failed).” We've discussed creating such an attestation as part of SLSA, but the community agreed that it was outside SLSA's scope. I'm curious whether in-toto would be interested in pursuing the idea.

IIUC, such an attestation is slightly outside of in-toto’s usual scope since it’s a statement about the attestor’s beliefs about the subject rather than metadata about the subject itself, but the framing allows us to move policy decisions further up the supply chain and away from admission control. For example, instead of gathering up a bundle of attestations for SLSA, vuln scans, etc and evaluating a policy about those attestations during admission control, you could check that policy into version control, evaluate it and produce a verification attestation, and then at admission control you have a much simpler policy that checks that there’s a passing verification attestation for the subject that uses the correct policy and comes from a trusted verifier.

As far as I can tell, in-toto doesn’t support inheritance, but I could imagine different sorts of inheritance attestations over time. The essential elements are the identity of the verifier, a policy URI, and whether the verification passed or failed. I’m not sure what to do about divergence between different verification attestations or whether it’s premature to think about now.

[JustinCappos]

I don't think this is out of scope. It's certainly possible to make an attestation that some process passed or failed and have it relate to that functionary's beliefs. From my perspective this is something we already support.

Do you feel that something is missing from our codebase or spec? Is there a reason this cannot be done off-the-shelf today? Is it a documentation issue instead?

(I'm punting on the inheritance aspects of your suggestion though.)

[kpk47]

Maybe this is a documentation issue? I think I'm mainly looking for validation of the idea and feedback on whether it's in or out of in-toto's scope. Inheritance is the hard part, so I suppose I'm implicitly asking if there's appetite for a change to an attestation layer or adding a new layer. Would it be better if I submitted a proposal for a new predicate?

cc @TomHennen

[kpk47]

There may also be a documentation problem, as all the examples I found discuss applying in-toto policies at admission control time. If in-toto supports moving policy checks earlier, then perhaps there could be an example of that pattern.

[adityasaky]

IIUC, such an attestation is slightly outside of in-toto’s usual scope since it’s a statement about the attestor’s beliefs about the subject rather than metadata about the subject itself

This is an interesting distinction. In my mental model, I apply that filter for all attestations as we always have to take some claims at face value as stated in the attestation.

For example, instead of gathering up a bundle of attestations for SLSA, vuln scans, etc and evaluating a policy about those attestations during admission control, you could check that policy into version control, evaluate it and produce a verification attestation, and then at admission control you have a much simpler policy that checks that there’s a passing verification attestation for the subject that uses the correct policy and comes from a trusted verifier.

IMO this is very much in-scope for in-toto. The verification workflow in in-toto's specification is recursive, resulting in an attestation (and in fact we're now exploring how we can use VSAs here) that is then verified by a "higher level" policy. This is commonly seen in the sublayout semantic but we've explored using verification summaries for low-resource contexts such as automotive software.

As far as I can tell, in-toto doesn’t support inheritance, but I could imagine different sorts of inheritance attestations over time. The essential elements are the identity of the verifier, a policy URI, and whether the verification passed or failed. I’m not sure what to do about divergence between different verification attestations or whether it’s premature to think about now.

The word "inheritance" is throwing me here (we've briefly thrown some inheritance-like ideas for predicates to complicate matters in my mind) but if my read of what you mean by it is right, I'm quite supportive of this.

There may also be a documentation problem, as all the examples I found discuss applying in-toto policies at admission control time. If in-toto supports moving policy checks earlier, then perhaps there could be an example of that pattern.

I don't know if I can speak for the attestation maintainers / other in-toto spec authors but for me "admission controller" is quite generic. A dependency review attestation (#151) would be verified prior to a library being admitted to the source control system, for example. What I'm getting at is that in my mind we already have this notion of verification at multiple points as demanded by context but perhaps it's not universal? If others have the same view, we should definitely make that more explicit in the docs.

[TomHennen]

Yeah I'd be supportive of something like this.

Is inheritance strictly necessary? I suspect we can probably meet existing needs without it, but maybe you have something in mind that won't work without it?

I think one thing that we'd discussed is 'moving' the VSA from SLSA to in-toto proper and making it more generic when we do so. If people want to attest to various different properties that were checked they could just go in what is now the verifiedLevels list, but could be renamed something like propertiesVerified.

[kpk47]

I think "inheritance" may be the wrong word. I'm thinking of something more like extensions in Protocol Buffers. Users could optionally add fields to the attestation to give supplemental information about the policy that they verified. Though the more I think about it, the more I think that such information would be better put in a separate attestation that's part of the same bundle. Would that be more idiomatic to in-toto?

I think one thing that we'd discussed is 'moving' the VSA from SLSA to in-toto proper and making it more generic when we do so. If people want to attest to various different properties that were checked they could just go in what is now the verifiedLevels list, but could be renamed something like propertiesVerified.

Thank you for bringing this up, and I apologize for not including the context. If in-toto had a "policy verification" attestation, then the SLSA community may encourage users to use that attestation instead of or in addition to VSAs.

[arewm]

I don't know if I can speak for the attestation maintainers / other in-toto spec authors but for me "admission controller" is quite generic. A dependency review attestation (https://github.com/in-toto/attestation/pull/151) would be verified prior to a library being admitted to the source control system, for example. What I'm getting at is that in my mind we already have this notion of verification at multiple points as demanded by context but perhaps it's not universal? If others have the same view, we should definitely make that more explicit in the docs.

The time required for a system to verify a policy based on metadata points is likely to be highly sensitive to the amount and type of data ingested. Taking the above example of validating a review attestation would likely be able to happen quickly as it is a single attestation which would contain the relevant data. A more complex policy based on multiple attestations (provenance, SBOMs, CI/CVE results, ...) can take more time especially if the sizes increase, if additional content needs to be retrieved (i.e. the source code itself), or if the data needs to be processed in order to make a policy decision.

While I agree that "admission controllers" are a generic term, there are certain situations where these controllers are more sensitive to quick verification. The example referenced above -- pulling in a dependency into a source control system -- will likely not result in an adverse experience as it might happen as part of a dedicated workflow or during an artifact's build. These are activities which individuals might not have a preconceived notion of occurring quickly. When images/pods are deployed to a kubernetes environment, however, users will often have a preconceived notion of the activity happening quickly -- resulting in a degraded experience.

Being able to run a complex process in an asynchronous fashion and attach an attestation with its result to an artifact would enable quick decisions in many situations, even when more complex policies are being evaluated.

[TomHennen]

Chatted about this at the maintainers meeting today.

1. We're generally for moving the VSA from SLSA to in-toto (and generalizing it in the process)
2. SLSA specific things, if needed, could be added via annotations of some sort.
3. We'd probably want the outstanding VSA PRs in the SLSA repo to be resolved one way or another before moving.
4. We'd want to avoid there being multiple different ways to represent the same thing (e.g. let's not have VSAs and something else that's very close to VSAs if we can avoid it).

[mikhailswift]

This is very relevant to our current work on Witness: https://github.com/testifysec/witness/issues/14 . We've had this issue in our backlog but are circling back to it in the short term.

After a quick read through the VSA spec, I think it would fit Witness's needs well. The only caveats are with number 2 in Tom's comment. Witness isn't currently asserting an artifact's SLSA level, though it may be able to in the future.

[TomHennen]

Note that I think @adityasaky also previously expressed some interested in having the in-toto verifier emit VSAs but I don't know if that's gone anywhere yet.

[adityasaky]

Indeed, but that work needs some more clarity on the status of VSA as a generic predicate type. https://github.com/in-toto/attestation-verifier is a great place to test out how VSAs can work for ITE-10/11 layouts.

[kpk47]

Glad to hear there's interest in moving VSA to in-toto. I'm working on closing out the remaining VSA PRs on the SLSA repo and should have that done shortly.

[TomHennen]

We've had some discussions about this internally and I put this together to share those thoughts here in case it helps: https://docs.google.com/presentation/d/1_1ikfSaja87-Qx2ev26DYdiNgS6AEvVFO92oSwNGvJ8/edit?usp=sharing Comments welcome.

[arewm]

This topic came up in the SLSA specification call today. Namely, the question was raised whether in-toto might need a new attestation type or whether SCAI would be able to be adapted to fulfil the needs of the SLSA verification summary attestations.

After reading the specification, it SCAI seems to be focused only on providing additional information and evidence (i.e. to potentially support future policy verification actions). @marcelamelara, do you have any additional context/proposal for how SCAI might be able to fit the needs of a VSA use case (i.e. as discussed today)?

[marcelamelara]

Based on @TomHennen slide deck, I've seen a couple of parallels between the fields in the simplified VSA format and the SCAI predicate fields:

• verifier <> producer in SCAI
• properties <> attributes in SCAI, where each verified attestation could be referenced as the target for a particular individual attribute. And since SCAI doesn't require the evidence field, it could be left empty.

The timeVerified field, or something similar, could possibly be added as an optional field to the SCAI spec. That said, the SCAI predicate, as currently specified, is quite generic, so there's definitely a need to dig a bit deeper into whether it provides all the needed semantics from a policy verification standpoint.

My plan for validating if/how SCAI could fit the needs of a VSA is to develop some examples for the types of policy checks/results that we want to be able to express. I can think of a few examples, but I definitely want to involve the community in this process so we can make sure we're covering a broader set of policy verification use cases.

[adityasaky]

(It's a bit late but I hope this comment is fairly clear...)

I recognize there's a lot of overlap in that VSA and SCAI both capture some set of (arbitrary-ish) attributes but I'm not sure that's necessarily enough to merge them into a single type. In my mind, there's a lot of value in seeing a VSA predicate type with the expectation that some verification in fact happened. While this can be communicated using SCAI, clients may have to do more work to distinguish a generic SCAI attestation and one specifically for policy verification.

I'll also highlight that SCAI is incredibly powerful precisely because it is generic and can be used for a variety of use cases, so I imagine we could argue for using SCAI in place of several other predicates, not just the VSA. Whether we should, IMO, goes back to the philosophy of having types in the first place to communicate something more specific with each one.

[marcelamelara]

Whether we should, IMO, goes back to the philosophy of having types in the first place to communicate something more specific with each one.

I completely agree with this. The main issue with using SCAI out of the box is that the predicateType doesn't convey the semantics. So, I've been thinking about whether we could introduce a way to sub-class existing predicates in cases like these. I can see this being useful not only in this particular scenario, but others as well. This idea probably veers into new ITE territory, so it may not be as immediately actionable.

[adityasaky]

So, I've been thinking about whether we could introduce a way to sub-class existing predicates in cases like these.

Without going too far off topic, I think this has come up before for sub-typing predicates, specifically for human review types. I could see this for a generic SBOM type that's a parent for SPDX and CDX, various subtypes for SCAI, etc. I'm definitely down to explore this more generally as a separate ITE, and what semantics we'd define for sub-typing in the attestation framework!

[kpk47]

The main issue with using SCAI out of the box is that the predicateType doesn't convey the semantics.

VSA also has that issue, which is part of the reason I stopped working on this PR. Attestations semantics are the hard part, not the format, so I felt like I was expending an undue amount of effort solving the easy part of a problem. I'm hopeful that whatever we end up using will comment on the policy's semantics in addition to whether or not the policy check passed.

So, I've been thinking about whether we could introduce a way to sub-class existing predicates in cases like these. I can see this being useful not only in this particular scenario, but others as well. This idea probably veers into new ITE territory, so it may not be as immediately actionable.

I also thought about sub-classing, but I couldn't convince myself that it really solved a problem unless it were paired with different expectations for how to define the types. If it's just as hard to create a sub-class of an existing predicate as it is to create a new predicate, why sub-class?

[laurentsimon]

I also thought about sub-classing, but I couldn't convince myself that it really solved a problem unless it were paired with different expectations for how to define the types. If it's just as hard to create a sub-class of an existing predicate as it is to create a new predicate, why sub-class?

+1. Sometimes a little bit of duplication is better than adding a lot of complexity.

[marcelamelara]

Attestations semantics are the hard part, not the format

100% and I appreciate you pointing this out again @kpk47 .

This brings me back to my earlier point about needing concrete policy examples and requirements. These will hopefully allow us to derive common semantics that consumers of policy verification attestations can reason about, and when we're ready to discuss formats again also find/develop a suitable one.