search

slsa-framework / slsa-github-generator

Language-agnostic SLSA provenance generation for Github Actions
Apache License 2.0
429 stars 129 forks source link

Feature: Should provenance include materials? #90

Open asraa opened 2 years ago

asraa commented 2 years ago

SLSA materials are:

materials array of objects, optional

The collection of artifacts that influenced the build including sources, dependencies, build tools, base images, and so on.

This is considered to be incomplete unless metadata.completeness.materials is true.

It seems like we currently include the source repo and digest, but should we also be including:

I don't see any examples of this, and since it's optional I understand it's not necessary but I would like to have a plan in mind. Go seems easy enough for dependencies because they are listed in the source anyway with hashes.

laurentsimon commented 2 years ago

related to slsa-framework/slsa-github-generator#89 and https://github.com/slsa-framework/slsa-github-generator-go/pull/16?

laurentsimon commented 2 years ago

We could start by adding the list of actions we use. We need to discuss what format this should look like. SLSA's material or SBOM reference?

asraa commented 2 years ago

I think we should include the SLSA material for actions. Unless we can re-use a GitHub workflow SBOM-maker somewhere.

I think the go builder should generate SBOM eventually, and we can embed the reference inside the provenance. Are there currently tools to generate SBOMs for go binaries? I feel like I heard rumors. Don't want to dupe work

ianlewis commented 2 years ago

Currently we include the GitHub repo and commit sha in the materials.

Do we need to do anything more?

laurentsimon commented 2 years ago

I think it was meant to include other materials, such as the dependencies. But maybe that will be replaced by SBOM eventually...

/cc @lumjjb any updates on this? We're interested in being a customer and see how we can integrate it in this repo

joshuagl commented 2 years ago

We could include details of the virtual-environment in the materials, but that's hard to link to in an immutable way.

ckotzbauer commented 2 years ago

Are there plans to add a generic extra-materials parameter to include other workflows or something else into the provenance? The philips-labs/slsa-provenance-action has this feature and it is very useful to add reusable workflows. You can see here how it works.

My use-case:

ianlewis commented 2 years ago

I think it was meant to include other materials, such as the dependencies. But maybe that will be replaced by SBOM eventually...

/cc @lumjjb any updates on this? We're interested in being a customer and see how we can integrate it in this repo

There are a couple of nuanced issues and here is my understanding:

  1. SLSA should include in materials the dependencies needed to reproduce the build. i.e. what was used as materials for the build. i.e. what the build system provided to the build when the build was kicked off.
  2. While the SBOM format is flexible enough to describe SLSA materials, SBOM is usually a bill of materials for the generated artifact which could be different from and not to be confused with the SLSA materials.

*1 and 2 might have significant overlap. To the extent that they are the same it is due to the limitations of tooling.

ianlewis commented 2 years ago

Are there plans to add a generic extra-materials parameter to include other workflows or something else into the provenance? The philips-labs/slsa-provenance-action has this feature and it is very useful to add reusable workflows. You can see here how it works.

Hi @ckotzbauer! What exactly is the extra-materials setting? In your example you include in extra-materials the uri of the github repo itself (github.com/ckotzbauer/actions-toolkit) so it seems like the materials are not exactly "extra" per se, at least in the context of GitHub Actions. Am I understanding that right?

slsa-github-generator will detect and add the user repo and digest automatically so you should be fine there. Are there other materials that you want to add?

ckotzbauer commented 2 years ago

The generator adds the repo and digest of the repository the action runs in (the place where the reusable action is called). That's totally fine, but in addition, I use other reusable workflows from other repositories which are not included.

Example: Project repository I want to release: "ckotzbauer/vulnerability-operator" (the concrete workflow lives here, but uses two reusable workflows):

Only the "slsa-framework/slsa-github-generator" is added as material automatically, but my build-workflow also affects the whole process and I cannot add it manually to the materials.

ianlewis commented 2 years ago

@ckotzbauer Thanks for the explanation. If I understand correctly you have a workflow A that calls workflow B and possibly C, D etc. and you want to capture all of A, B, C, and D in the provenance? Am I understanding that right?

Only the "slsa-framework/slsa-github-generator" is added as material automatically, but my build-workflow also affects the whole process and I cannot add it manually to the materials.

I think there may be some confusion because the materials generated by the generic generator contains the repository that triggered the workflow and not the reusable workflow. Here's an example from my test repo (I think this would be ckotzbauer/vulnerability-operator in your case).

    "materials": [
      {
        "uri": "git+https://github.com/ianlewis/actions-test@refs/tags/v0.0.9",
        "digest": {
          "sha1": "e491e4b2ce5bc76fb103729b61b04d3c46d8a192"
        }
      }
    ]

The provenance should also contain a link to the top level workflow that was run in the invocation that is separate from the materials. From this reference you should be able to track down what other workflows were run and, along with the environment part of the invocation should technically be enough to reproduce the build and capture the whole process.

  "invocation": {
      "configSource": {
        "uri": "git+https://github.com/ianlewis/actions-test@refs/heads/main.git",
        "digest": {
          "sha1": "e491e4b2ce5bc76fb103729b61b04d3c46d8a192"
        },
        "entryPoint": ".github/workflows/generic-container.yml"
      },
      "parameters": {},
      "environment": {
        ...
      }
  ...

In your case I think this should point to .github/workflows/toolkit-release-goreleaser.yml in the ckotzbauer/vulnerability-operator repo. So someone wanting to put together what the build did should be able to check out the code at the digest referenced in the materials and (separately) inspect .github/workflows/toolkit-release-goreleaser.yml file at the digest referenced in the invocation to understand how it was kicked off and what build code was executed (in the case of GitHub actions they are the same repo and digest).

Where the generic generator does refer to itself is in the builder.id and buildType. Maybe you were thinking of that?

    "builder": {
      "id": "https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@refs/tags/v1.2.0"
    },
    "buildType": "https://github.com/slsa-framework/slsa-github-generator/generic@v1",

With all that said, I'm not sure it's the level of granularity we want and I'm not sure we could effectively capture all the information in a better way than the workflow files themselves could capture.

One issue I do see is that many times actions and workflows are referred to by mutable tags or without a version at all and get run at HEAD of their default branch. This could make it hard to know exactly which version of a reusable workflow was run. I'm not sure there is a way to capture that information accurately and reliably though. I'm a bit wary of having user workflows add things manually to the provenance via inputs because user code is not supposed to be able to modify fields such as the materials.

ckotzbauer commented 2 years ago

Thanks for your detailed explanation @ianlewis.

If I understand correctly you have a workflow A that calls workflow B and possibly C, D etc. and you want to capture all of A, B, C, and D in the provenance? Am I understanding that right?

Not A calls B and B calls C; A calls first B and then C afterwards (when B finished).

In regards to invocation, environment and materials I saw the same references as you. As long as the current state conforms with the SLSA requirements for level 3, I'm totally fine with that. Maybe my understanding from the requirements were a bit too strict, that all build-workflows have to be part of the materials section.

laurentsimon commented 2 years ago

IIUC, the re-usable workflow at https://github.com/ckotzbauer/actions-toolkit/blob/main/.github/workflows/toolkit-release-goreleaser.yml is used by all the org projects to release artifacts. The generic generator does not allow specifying additional materials / re-usable workflow that were called prior to calling the generic generator:

  1. Since the re-usable workflow relies on an action, @ckotzbauer wanted to add it to the list of materials. I think @ianlewis is right that materials should be components that went into the build, not components that were used to build. The action could be reported in the buildConfig instead, saying that it was run with certain inputs. The generic generator does not support this, though.
  2. The re-usable workflow https://github.com/ckotzbauer/actions-toolkit/blob/main/.github/workflows/toolkit-release-goreleaser.yml won't be present in the provenance we generate, because it's not part of the "GitHub context", so the entrypoint will point to the developer workflow, not the toolkit-release-goreleaser.yml. @ckotzbauer, I think you're asking to be able to indicate that the caller was your re-usable workflow in the provenance, correct?

Fyi, we're working on this feature which may be what you're interested in https://github.com/slsa-framework/slsa-github-generator/issues/763#issuecomment-1258373620

ckotzbauer commented 2 years ago

That's exactly what I'm looking for. Thanks @laurentsimon. :+1: