oracle / macaron

Macaron is an extensible supply-chain security analysis framework from Oracle Labs that supports a wide range of build systems and CI/CD services. It can be used to prevent supply chain attacks, detect malicious Python packages, or check conformance to frameworks, such as SLSA. Documentation:
https://oracle.github.io/macaron/
Universal Permissive License v1.0
137 stars 23 forks source link

Unintuitive behavior of the provenance expectation check #595

Open nathanwn opened 10 months ago

nathanwn commented 10 months ago

Reproduction

We use the macaron analyze command under the use case of "user providing both a PackageURL to identify a software component and a repo URL explicitly".

macaron analyze \
  --package-url pkg:maven/io.micronaut/micronaut-test@4.1.1 \
  --repo-path https://github.com/micronaut-projects/micronaut-test \
  --digest 0a43363f7562534063e06e3f2a328f09a066b547 \
  --skip-deps \

Macaron then does not recognize the following expectation to correspond to the software component being analyzed:

{
  target: "pkg:maven/io.micronaut/micronaut-test@4.1.1",
  predicate: {
    builder: {
      id: =~"^https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml.*"
    }
  }
}

However, after removing the version part from the PURL in the target field, Macaron now recognizes this expectation to correspond to the software component.

 {
-  target: "pkg:maven/io.micronaut/micronaut-test@4.1.1",
+  target: "pkg:maven/io.micronaut/micronaut-test",

This is undesirable from a user's perspective because it is not clear why the same PURL can be specified in the analyze command, but not in the expectation.

Cause

Macaron stores expectations for all analysis targets in a dictionary, in which each key is a PURL identifying a software component, and each value is the corresponding provenance expectation for that software component.

Before a pair of (PURL, expectation) is added to this dictionary, the version part of the PURL (starting from the @ character), is stripped away from the PURL.

Other unexpected details

behnazh-w commented 10 months ago

Thanks for reporting this issue. Part of this behavior is actually intended:

However, after removing the version part from the PURL in the target field, Macaron now recognizes this expectation to correspond to the software component.

We don't want to require the version string to be specified in the expectation. Otherwise, the expectation file needs to be updated for each artifact version.

nathanwn commented 10 months ago

I wonder if we could provide a regex option to let users decide if they want to keep the target field version-agnostic. It may look like this, where =~ is the Cue regex matching operator:

{
  target: =~"pkg:maven/io.micronaut/micronaut-test.*",
  ...

This may require changes to how the expectation is evaluated.