javascript-lock cataloger not picking up licenses in node_modules package.json files

jeremytbrun commented 1 year ago

What happened: I have a package-lock v3 file and have ran npm install. I've verified all 3rd party packages are installed to the local node_modules folder. When running this I get an syft-json SBOM but all "licenses" nodes are empty []

syft .\ -o syft-json=.\syft.json --catalogers javascript-lock

What you expected to happen: Based on #1910 and #1548 I'd expect there to be license data inside of the syft-json SBOM because syft should have pulled that out of individual \node_modules\packageName\package.json files.

Steps to reproduce the issue:

Anything else we need to know?:

Environment:

Output of syft version: Application: syft Version: 0.94.0 BuildDate: 2023-10-20T17:21:07Z GitCommit: 8f6bdde6662aa8050a71eadbdb7bd5a3b079a56d GitDescription: v0.94.0 Platform: windows/amd64 GoVersion: go1.21.3 Compiler: gc
OS (e.g: cat /etc/os-release or similar): Windows 11 Enterprise 21H2

tgerla commented 1 year ago

Hi @jeremytbrun, would it be possible for you to share the package-lock file you are using, so we can reproduce? Thanks!

jeremytbrun commented 1 year ago

Hi @jeremytbrun, would it be possible for you to share the package-lock file you are using, so we can reproduce? Thanks!

package-lock.json

tgerla commented 1 year ago

I gave it a shot on my system (MacOS Sonoma, Syft 0.94) and I'm getting the appropriate license information in the JSON SBOM:

  {
   "id": "555a079bf19e0ea1",
   "name": "@angular-devkit/core",
   "version": "15.2.9",
   "type": "npm",
   "foundBy": "javascript-lock-cataloger",
   "locations": [
    {
     "path": "/package-lock.json",
     "annotations": {
      "evidence": "primary"
     }
    }
   ],
   "licenses": [
    {
     "value": "MIT",
     "spdxExpression": "MIT",
     "type": "declared",
     "urls": [],
     "locations": [
      {
       "path": "/package-lock.json",
       "annotations": {
        "evidence": "primary"
       }
      }
     ]
    }
   ],
...

Do you mind also attaching the resulting SBOM that you're getting? I could compare the results and see if anything jumps out.

jeremytbrun commented 1 year ago

I gave it a shot on my system (MacOS Sonoma, Syft 0.94) and I'm getting the appropriate license information in the JSON SBOM:
  {
   "id": "555a079bf19e0ea1",
   "name": "@angular-devkit/core",
   "version": "15.2.9",
   "type": "npm",
   "foundBy": "javascript-lock-cataloger",
   "locations": [
    {
     "path": "/package-lock.json",
     "annotations": {
      "evidence": "primary"
     }
    }
   ],
   "licenses": [
    {
     "value": "MIT",
     "spdxExpression": "MIT",
     "type": "declared",
     "urls": [],
     "locations": [
      {
       "path": "/package-lock.json",
       "annotations": {
        "evidence": "primary"
       }
      }
     ]
    }
   ],
...
Do you mind also attaching the resulting SBOM that you're getting? I could compare the results and see if anything jumps out.

Glad it works. I will say the only change I made to the file before providing it was removing some package references stored in private repository cuz I thought that would break it for you. The private repository is in Azure Artifacts. Not sure if you could try something with a private repository.

I will try to get the SBOM to you but it might be a couple days because I'm on vacation.

jeremytbrun commented 1 year ago

I gave it a shot on my system (MacOS Sonoma, Syft 0.94) and I'm getting the appropriate license information in the JSON SBOM:
  {
   "id": "555a079bf19e0ea1",
   "name": "@angular-devkit/core",
   "version": "15.2.9",
   "type": "npm",
   "foundBy": "javascript-lock-cataloger",
   "locations": [
    {
     "path": "/package-lock.json",
     "annotations": {
      "evidence": "primary"
     }
    }
   ],
   "licenses": [
    {
     "value": "MIT",
     "spdxExpression": "MIT",
     "type": "declared",
     "urls": [],
     "locations": [
      {
       "path": "/package-lock.json",
       "annotations": {
        "evidence": "primary"
       }
      }
     ]
    }
   ],
...
Do you mind also attaching the resulting SBOM that you're getting? I could compare the results and see if anything jumps out.
Glad it works. I will say the only change I made to the file before providing it was removing some package references stored in private repository cuz I thought that would break it for you. The private repository is in Azure Artifacts. Not sure if you could try something with a private repository.

I will try to get the SBOM to you but it might be a couple days because I'm on vacation.

Do you mind sharing the exact command syntax you used?

tgerla commented 1 year ago

Sure, I am not an NPM expert so I wasn't quite sure what I was doing with the package lock file, but I ended up doing:

$ npm ci --legacy-peer-deps

This got the node_modules installed and then I ran:

$ syft . -o syft-json --catalogers javascript-lock  > sbom.json

Hope this helps...happy to dig back in when you are back from vacation.

jeremytbrun commented 1 year ago

Sure, I am not an NPM expert so I wasn't quite sure what I was doing with the package lock file, but I ended up doing:
$ npm ci --legacy-peer-deps
This got the node_modules installed and then I ran:
$ syft . -o syft-json --catalogers javascript-lock  > sbom.json
Hope this helps...happy to dig back in when you are back from vacation.

I went ahead and pulled the exact same package-lock.json file I supplied earlier and ran the same commands you did. The resulting sbom.json does not have license data like it does if you run it.

{
   "id": "f0d37ead64b9e42b",
   "name": "@angular-devkit/core",
   "version": "15.2.9",
   "type": "npm",
   "foundBy": "javascript-lock-cataloger",
   "locations": [
    {
     "path": "\\package-lock.json",
     "annotations": {
      "evidence": "primary"
     }
    }
   ],
   "licenses": [],
...

This is with version Syft v0.94.0. Also here is the output of npm version on my development machine.

{
  npm: '9.8.1',
  node: '18.18.2',
  acorn: '8.10.0',
  ada: '2.6.0',
  ares: '1.19.1',
  brotli: '1.0.9',
  cldr: '43.1',
  icu: '73.2',
  llhttp: '6.0.11',
  modules: '108',
  napi: '9',
  nghttp2: '1.57.0',
  nghttp3: '0.7.0',
  ngtcp2: '0.8.1',
  openssl: '3.0.10+quic',
  simdutf: '3.2.14',
  tz: '2023c',
  undici: '5.26.3',
  unicode: '15.0',
  uv: '1.44.2',
  uvwasi: '0.0.18',
  v8: '10.2.154.26-node.26',
  zlib: '1.2.13.1-motley'
}

tgerla commented 1 year ago

Hi @jeremytbrun, I think the difference might be my macOS-based system versus your Windows-based system. We will add this to our backlog to reproduce and fix when we are able.

mc-alt commented 8 months ago

I think I might also be affected by this. The licenses section of my javascript outputs are empty, and I am running on Windows

kevin-kortum-trustedshops commented 3 weeks ago

I might throw my hat into this as well.

Running both syft --from dir node_modules --output spdx-json=sbom.json or syft --from file yarn.lock --select-catalogers javascript --output spdx-json=sbom.json

Will not pick any licenses, although in the node_modules folders most packages provide a license file.

anchore / syft

javascript-lock cataloger not picking up licenses in node_modules package.json files #2260