search

CycloneDX / cdxgen

Creates CycloneDX Bill of Materials (BOM) for your projects from source and container images. Supports many languages and package managers. Integrate in your CI/CD pipeline with automatic submission to Dependency Track server. Slack: https://cyclonedx.slack.com/archives/C04NFFE1962
https://cyclonedx.github.io/cdxgen/
Apache License 2.0
493 stars 142 forks source link

cdxgen fails to find all dependencies #1094

Open emcfins opened 1 month ago

emcfins commented 1 month ago

When viewing the dependency graph for Connected Mobility Solution on AWS, it shows a total of 7,484. But when I clone the repo and run CDXGEN_DEBUG_MODE=debug FETCH_LICENSE=true cdxgen -t universal --spec-version 1.4 -o bom.json, the components section of the bom.json file, the components section only has 4289.

For example, I find some dependencies missing from the sbom that are defined in the lock file here

What am I doing wrong?

Thank you

prabhu commented 1 month ago

@emcfins Have you tried running with -t python --deep, since it appears like a Python project. We have some logic in place, where -t universal would try to go broader rather than deeper into each language. You may have to run cdxgen from within a python virtual environment with all the packages installed for best results.

emcfins commented 1 month ago

We are using the same tooling for multiple projects - we have a mix of projects of different languages and some projects are a mix of languages so we use -t universal to cover all our requirements.

prabhu commented 1 month ago

@emcfins have you tried without specifying any type?

emcfins commented 1 month ago

Oh - no I haven't. I'll give that a shot. Thank you!

emcfins commented 1 month ago

Not specifying didn't work. Unfortunately, that returned 4287 components

prabhu commented 1 month ago

@emcfins, this needs a bit more triaging. Could you create a folder with just this file and test cdxgen?

The logic to parse a pipfile.lock is here. Is there any bug in the logic?

You can add a unit test similar to this, once you have identified some fixes.

prabhu commented 1 month ago

@aryan-rajoria could you kindly work with @emcfins, since this is false negatives?

emcfins commented 1 month ago

So I did some digging. When I run cdxgen locally, I noticed the output:

Parsing /Users/erinmc/projects/aws_solutions/connected-mobility/Connected-mobility-solution-on-aws/source/modules/backstage/yarn.lock
About to fetch license information for 3237 packages in parseYarnLock
Found 4226 npm packages at .
Found 61 python packages at .
Found 0 ruby packages at .
Found 0 cpp packages at .
BOM includes 4287 components and 3979 dependencies after dedupe
BOM includes 4289 unfiltered components 3979 dependencies so far
BOM includes 4289 components and 3979 dependencies after dedupe

Is it possible that cdxgen is just looking for a single Pipfile rather than all Pipfiles?

prabhu commented 1 month ago

@emcfins Nice find. I think it only looks for a single file and performs a single pipenv install.

https://github.com/CycloneDX/cdxgen/blob/4e6f8da04334fe6babd8fd627bf1b925e41b7749/index.js#L2473

https://github.com/CycloneDX/cdxgen/blob/4e6f8da04334fe6babd8fd627bf1b925e41b7749/index.js#L2625

prabhu commented 1 month ago

@emcfins Let me know if you would like to sponsor or contribute this feature.

emcfins commented 1 month ago

I'm happy to sponsor - what does it take?

prabhu commented 1 month ago

@emcfins, Thank you so much! Our donation link is here

One of us will find time and implement this.