search

CycloneDX / cdxgen

Creates CycloneDX Bill of Materials (BOM) for your projects from source and container images. Supports many languages and package managers. Integrate in your CI/CD pipeline with automatic submission to Dependency Track server. Google chat: https://chat.google.com/room/AAAA6l2dO60?cls=7
https://cyclonedx.github.io/cdxgen/
Apache License 2.0
568 stars 157 forks source link

[BUG] cdxgen -o bom.json returns error on Trivy repo #954

Open almaz045 opened 6 months ago

almaz045 commented 6 months ago

When I run this command "cdxgen -o bom.json" on other GO repos it's work fine. But when I run on Trivy repo, it's return errors. 

trivy-main$ cdxgen -o bom.json

Unable to parse trivy-main/pkg/fanal/analyzer/language/nodejs/npm/testdata/sad/package-lock.json without legacy peer dependencies. Retrying ...
Unable to parse trivy-main/pkg/fanal/analyzer/language/nodejs/npm/testdata/sad/package-lock.json in legacy and non-legacy mode. The resulting SBOM would be incomplete.
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/requirements
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/happy
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/broken
Fallback to executing mvn dependency:tree -DoutputFile=/tmp/cdxmvn-NUYavo/mvn-tree.txt

Falling back to manual pom.xml parsing. The result would be incomplete!
/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:652
    er = new Error(er)
         ^

<ref *1> Error: Text data outside of root node.
Line: 0
Column: 2
Char: }
    at error (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:652:10)
    at strictFail (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:678:7)
    at SAXParser.write (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:1036:15)
    at module.exports (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/xml-js/lib/xml2js.js:346:12)
    at parsePom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/utils.js:1654:19)
    at createJavaBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:1244:27)
    at createMultiXBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:4719:21)
    at async createBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:5671:16)
    at async file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/bin/cdxgen.js:431:20 {
  note: [Circular *1]
}

Node.js v18.16.0

$ cdxgen --version
10.2.6
prabhu commented 6 months ago

@almaz045, can you install nodejs >= 20 since v10 wouldn't work with node 18.

https://github.com/CycloneDX/cdxgen/blob/master/package.json#L51

almaz045 commented 6 months ago 
cdxgen -o bom.json

Unable to parse trivy-main/pkg/fanal/analyzer/language/nodejs/npm/testdata/sad/package-lock.json without legacy peer dependencies. Retrying ...
Unable to parse trivy-main/pkg/fanal/analyzer/language/nodejs/npm/testdata/sad/package-lock.json in legacy and non-legacy mode. The resulting SBOM would be incomplete.
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/requirements
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/happy
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/broken
Fallback to executing mvn dependency:tree -DoutputFile=/tmp/cdxmvn-JXKdy4/mvn-tree.txt

Falling back to manual pom.xml parsing. The result would be incomplete!
/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:652
    er = new Error(er)
         ^

<ref *1> Error: Text data outside of root node.
Line: 0
Column: 2
Char: }
    at error (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:652:10)
    at strictFail (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:678:7)
    at SAXParser.write (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/sax/lib/sax.js:1036:15)
    at module.exports (/usr/local/lib/node_modules/@cyclonedx/cdxgen/node_modules/xml-js/lib/xml2js.js:346:12)
    at parsePom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/utils.js:1654:19)
    at createJavaBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:1244:27)
    at createMultiXBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:4719:21)
    at async createBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:5671:16)
    at async file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/bin/cdxgen.js:431:20 {
  note: [Circular *1]
}

Node.js v20.12.0

cdxgen --version
10.2.6
prabhu commented 6 months ago

@almaz045 can you also pass -t go. I think there is some invalid pom.xml in that repo that is causing the failure.

almaz045 commented 6 months ago

@almaz045 can you also pass -t go. I think there is some invalid pom.xml in that repo that is causing the failure.

Yes, with -t go works fine. Just with other GO repos it's work fine without flag "-t go" and I decided to highlight this moment, maybe will be useful.

almaz045 commented 6 months ago

@almaz045 can you also pass -t go. I think there is some invalid pom.xml in that repo that is causing the failure.

I inserted pom.xml into the online xml validator and it did not find any errors in it

prabhu commented 6 months ago

trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/broken

Appears like a broken test case.

almaz045 commented 6 months ago

Yes, you were right. But another errors now...

cdxgen -o bom.json

Unable to parse trivy-main/pkg/fanal/analyzer/language/nodejs/npm/testdata/sad/package-lock.json without legacy peer dependencies. Retrying ...
Unable to parse trivy-main/pkg/fanal/analyzer/language/nodejs/npm/testdata/sad/package-lock.json in legacy and non-legacy mode. The resulting SBOM would be incomplete.
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/requirements
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/pkg/fanal/analyzer/language/java/pom/testdata/happy
Executing 'mvn org.cyclonedx:cyclonedx-maven-plugin:2.7.11:makeAggregateBom -DoutputName=bom -DincludeTestScope=true -q' in trivy-main/integration/testdata/fixtures/repo/pom
Poetry install has failed. Setup and activate the poetry virtual environment and re-run cdxgen.
go unexpectedly didn't return any output. Check if the correct version of golang is installed.
go unexpectedly didn't return any output. Check if the correct version of golang is installed.
go unexpectedly didn't return any output. Check if the correct version of golang is installed.
go unexpectedly didn't return any output. Check if the correct version of golang is installed.
go unexpectedly didn't return any output. Check if the correct version of golang is installed.
go unexpectedly didn't return any output. Check if the correct version of golang is installed.
Manually parsing go.mod files. The resultant BOM would be incomplete.
undefined:1
{

SyntaxError: Expected property name or '}' in JSON at position 1
    at JSON.parse (<anonymous>)
    at createPHPBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:4136:35)
    at createMultiXBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:4807:15)
    at async createBom (file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/index.js:5671:16)
    at async file:///usr/local/lib/node_modules/@cyclonedx/cdxgen/bin/cdxgen.js:431:20

Node.js v20.12.0

go version
go version go1.22.0 linux/amd64

It would be nice if we could handle errors when parsing pom.xml, go.mod, poetry.toml, etc. For example, if we see that pom.xml is incorrect, then we notify about it as WARN and skip this file, moving on.

almaz045 commented 6 months ago

I installed poetry but I still keep getting the error

poetry --version
Poetry (version 1.8.2)

find . -name "pyproject.toml"
./integration/testdata/fixtures/repo/poetry/pyproject.toml
./pkg/fanal/analyzer/language/python/poetry/testdata/wrong-pyproject/pyproject.toml
./pkg/fanal/analyzer/language/python/poetry/testdata/happy/pyproject.toml
prabhu commented 6 months ago

@almaz045, happy to keep this ticket open and look for funding. We actually have a donation page to receive funding to hire real engineers to work on our projects. We pay full market rate to the contributors. Fingers crossed!

https://owasp.org/donate/?reponame=www-project-cyclonedx&title=OWASP+CycloneDX