Capture licenses for all packages

kzantow commented 3 months ago

Syft should be able to include license information for packages it finds. Sometimes this information is present in the metadata on disk, other times it is only available by some remote source. This is an uber-issue about capturing licenses for all packages. Each ecosystem will likely have a different mechanism of capturing license information.

Ecosystems:

[ ] Apk
[ ] Dpkg
[ ] ALPM
[ ] binary (https://github.com/anchore/syft/issues/2765)
[ ] Conan (conan.lock)
[ ] Conan (conanfile.txt)
[ ] Dart (pubspec.lock)
[ ] .NET (deps.json) https://github.com/anchore/syft/issues/1227
[ ] .NET (from binary)
[ ] Elixir (mix.lock)
[ ] Erlang (rebar.lock)
[ ] Github actions workflows (workflows using actions)
[x] Golang (go.mod)
[x] Golang (binary)
[ ] Haskell (stack.yaml)
[ ] Haskell (stack.yaml.lock)
[ ] Haskell (cabal.project.freeze)
[ ] Java (nested jars)
[ ] Java (pom.xml)
[ ] Java (gradle.lockfile)
[ ] Javascript (package.json) https://github.com/anchore/syft/issues/2330
[ ] Javascript (package-lock.json)
[ ] Javascript (yarn.lock)
[ ] Javascript (pnpm-lock.yaml)
[ ] Kernel modules
[ ] Nix (store)
[ ] PHP (installed.json)
[ ] PHP (composer.lock)
[ ] Portage (contents file)
[ ] Python (poetry.lock)
[ ] Python (setup.py)
[ ] Python (requirements.txt)
[ ] Python (Pipfile.lock)
[ ] Python (egg/wheel metadata)
[ ] R (description file)
[ ] RPM (db)
[ ] RPM (rpm file)
[ ] Ruby (gemfile.lock)
[ ] Ruby (specifications gemspec)
[ ] Rust (cargo.lock)
[ ] Rust (binary)
[ ] SBOM
[ ] Swift (package.resolved)
[ ] Swift (Podfile.lock)

Some related issues:

mykaul commented 3 months ago

@kzantow - why is Go marked as checked? How do we get the license of Go modules?

tgerla commented 3 months ago

@mykaul check out the golang section of the Syft configuration file: https://github.com/anchore/syft/wiki/configuration -- there are two settings, search-local-mod-cache-licenses and search-remote-licenses that can be enabled to retrieve license data.

mykaul commented 3 months ago

@mykaul check out the golang section of the Syft configuration file: https://github.com/anchore/syft/wiki/configuration -- there are two settings, search-local-mod-cache-licenses and search-remote-licenses that can be enabled to retrieve license data.

Thanks! I think what tricked me is that by default (syft-text?) you do not see the license, so I did not even bother to look further. Very helpful, thanks again.

Annamikhlin commented 3 months ago

Any idea why I get different output report when scanning the same package with remote search licenses: true for Go modules on different machines?

On one machine - licenses info is presented:

{
      "id": "5a2f10fe8c37697d",
      "name": "github.com/alecthomas/units",
      "version": "v0.0.0-20211218093645-b94a6e3cc137",
      "type": "go-module",
      "foundBy": "go-module-binary-cataloger",
      "locations": [
        {
          "path": "/opt/scylladb/node_exporter/node_exporter",
          "layerID": "sha256:2f9c5474dfcad40cab09d578dfc5c919a38e0125c99501a8d7abb8540ef14188",
          "accessPath": "/opt/scylladb/node_exporter/node_exporter",
          "annotations": {
            "evidence": "primary"
          }
        }
      ],
      "licenses": [
        {
          "value": "MIT",
          "spdxExpression": "MIT",
          "type": "concluded",
          "urls": [],
          "locations": [
            {
              "path": "github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137/COPYING",
              "accessPath": "github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137/COPYING"
            }
          ]
        }
      ],
      "language": "go",
      "cpes": [
        {
          "cpe": "cpe:2.3:a:alecthomas:units:v0.0.0-20211218093645-b94a6e3cc137:*:*:*:*:*:*:*",
          "source": "syft-generated"
        }
      ],
      "purl": "pkg:golang/github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137",
      "metadataType": "go-module-buildinfo-entry",
      "metadata": {
        "goCompiledVersion": "go1.21.4",
        "architecture": "amd64",
        "h1Digest": "h1:s6gZFSlWYmbqAuRjVTiNNhvNRfY2Wxp9nhfyel4rklc=",
        "mainModule": "github.com/prometheus/node_exporter"
      }
    },

On another machine the license info is missing:

    {
      "id": "67d84fc35f370e95",
      "name": "github.com/alecthomas/units",
      "version": "v0.0.0-20211218093645-b94a6e3cc137",
      "type": "go-module",
      "foundBy": "go-module-binary-cataloger",
      "locations": [
        {
          "path": "/opt/scylladb/node_exporter/node_exporter",
          "layerID": "sha256:2f9c5474dfcad40cab09d578dfc5c919a38e0125c99501a8d7abb8540ef14188",
          "accessPath": "/opt/scylladb/node_exporter/node_exporter",
          "annotations": {
            "evidence": "primary"
          }
        }
      ],
      "licenses": [],
      "language": "go",
      "cpes": [
        {
          "cpe": "cpe:2.3:a:alecthomas:units:v0.0.0-20211218093645-b94a6e3cc137:*:*:*:*:*:*:*",
          "source": "syft-generated"
        }
      ],
      "purl": "pkg:golang/github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137",
      "metadataType": "go-module-buildinfo-entry",
      "metadata": {
        "goCompiledVersion": "go1.21.4",
        "architecture": "amd64",
        "h1Digest": "h1:s6gZFSlWYmbqAuRjVTiNNhvNRfY2Wxp9nhfyel4rklc=",
        "mainModule": "github.com/prometheus/node_exporter"
      }
    },

both machines are installed with Fedora release 37

mykaul commented 3 months ago

Very the configuration file is identical and accessible in both machines. Perhaps run syft with debug will show it.

Annamikhlin commented 3 months ago

Any idea why I get different output report when scanning the same package with remote search licenses: true for Go modules on different machines?

On one machine - licenses info is presented:

{
      "id": "5a2f10fe8c37697d",
      "name": "github.com/alecthomas/units",
      "version": "v0.0.0-20211218093645-b94a6e3cc137",
      "type": "go-module",
      "foundBy": "go-module-binary-cataloger",
      "locations": [
        {
          "path": "/opt/scylladb/node_exporter/node_exporter",
          "layerID": "sha256:2f9c5474dfcad40cab09d578dfc5c919a38e0125c99501a8d7abb8540ef14188",
          "accessPath": "/opt/scylladb/node_exporter/node_exporter",
          "annotations": {
            "evidence": "primary"
          }
        }
      ],
      "licenses": [
        {
          "value": "MIT",
          "spdxExpression": "MIT",
          "type": "concluded",
          "urls": [],
          "locations": [
            {
              "path": "github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137/COPYING",
              "accessPath": "github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137/COPYING"
            }
          ]
        }
      ],
      "language": "go",
      "cpes": [
        {
          "cpe": "cpe:2.3:a:alecthomas:units:v0.0.0-20211218093645-b94a6e3cc137:*:*:*:*:*:*:*",
          "source": "syft-generated"
        }
      ],
      "purl": "pkg:golang/github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137",
      "metadataType": "go-module-buildinfo-entry",
      "metadata": {
        "goCompiledVersion": "go1.21.4",
        "architecture": "amd64",
        "h1Digest": "h1:s6gZFSlWYmbqAuRjVTiNNhvNRfY2Wxp9nhfyel4rklc=",
        "mainModule": "github.com/prometheus/node_exporter"
      }
    },

On another machine the license info is missing:

    {
      "id": "67d84fc35f370e95",
      "name": "github.com/alecthomas/units",
      "version": "v0.0.0-20211218093645-b94a6e3cc137",
      "type": "go-module",
      "foundBy": "go-module-binary-cataloger",
      "locations": [
        {
          "path": "/opt/scylladb/node_exporter/node_exporter",
          "layerID": "sha256:2f9c5474dfcad40cab09d578dfc5c919a38e0125c99501a8d7abb8540ef14188",
          "accessPath": "/opt/scylladb/node_exporter/node_exporter",
          "annotations": {
            "evidence": "primary"
          }
        }
      ],
      "licenses": [],
      "language": "go",
      "cpes": [
        {
          "cpe": "cpe:2.3:a:alecthomas:units:v0.0.0-20211218093645-b94a6e3cc137:*:*:*:*:*:*:*",
          "source": "syft-generated"
        }
      ],
      "purl": "pkg:golang/github.com/alecthomas/units@v0.0.0-20211218093645-b94a6e3cc137",
      "metadataType": "go-module-buildinfo-entry",
      "metadata": {
        "goCompiledVersion": "go1.21.4",
        "architecture": "amd64",
        "h1Digest": "h1:s6gZFSlWYmbqAuRjVTiNNhvNRfY2Wxp9nhfyel4rklc=",
        "mainModule": "github.com/prometheus/node_exporter"
      }
    },

both machines are installed with Fedora release 37

seems it is related to the same issue https://github.com/anchore/syft/issues/2798 The creation of $HOME/go/pkg/mod directory - solved the issue waiting for https://github.com/anchore/syft/pull/2852 for official fix

anchore / syft

Capture licenses for all packages #2861