Propagate vulnerabilities from child project to parent

ghost commented 3 years ago

I searched open and closed issue and didn't find anything matching. I use release of one project as a dependency in another project.

Current Behavior:

So, project a is tracked with its dependencies in DT. Project b has its dependencies inventorized and one of its dependency is project a. Project b does not get vulnerability information from project a. Because the component "project b" does not resolve to the project b in DT.

Proposed Behavior:

I suggest that it is already kind of possible to do that, if you would set up a mechanism that creates a matchable component in DT to every project release. Furthermore you need to set CPE that would be resolved by the internal database and update the vulnerabilities to that matchable component as vulnerabilities are discovered in the corresponding project. DT should provide a mechanism to set a component to an internal DT project, without hacking it manually.

stevespringett commented 3 years ago

Excellent suggestion

lihaoran93 commented 3 years ago

Current: If the dependency of Project B contains the dependency of project A, will the Bom of project B contain the dependency of project A, and the vulnerability analysis will cover all of them?

lihaoran93 commented 3 years ago

Instead, I encountered a situation where Project B contained Project A, but I did not want the vulnerability information of Project A to be shown in Project B, because it was up to different teams to deal with these vulnerabilities，I am struggling with how to eliminate the vulnerability information of Project A

stevespringett commented 3 years ago

For this enhancement, I think it's important to realize there is separate accountability - each project being accountable for their own vulnerability information. However, I think that it's also important to inherit the risk (including risk scores and policy violations) from the other project.

lihaoran93 commented 3 years ago

Current: If the dependency of Project B contains the dependency of project A, will the Bom of project B contain the dependency of project A, and the vulnerability analysis will cover all of them?

Is this description correct? Is there A way to exclude the vulnerability information of Project A? Bom information contains all components by default, but does not identify which components are from project A. If so, should it be modified from the BOM generation side (CyclonedX-Maven)?

stevespringett commented 3 years ago

In order for this to work, #280 will likely need to be implemented so that DT can know that another project was responsible for bringing in vulnerable dependencies.

ghost commented 3 years ago

r, I think that it's also import

I agree with that.

I'd like to mention that my initial thought was to not resolve the dependencies from the project a in project b, but to rather have a component project a in the bom of project b. This does already work, but the connection between Project A and the component project a is missing.

StyleT commented 2 years ago

Hello from Namecheap! We're currently looking at DependencyTrack as a tool to track vulnerabilities in OS deps within project portfolio and this feature is very important for us. As we want to understand impact of the vulnerability with consideration of the inter-project dependencies. Why it's important: If lib A is vulnerable and it's used in half of the apps we have on production - this is much bigger alarm than equal vulnerability in single app.

Would be nice to hear if there are any updates since 2020?

stevespringett commented 2 years ago

@StyleT this is a p2 enhancement request that doesn't have a target version, therefore it is not on the short-term roadmap. As an open source project, pull requests are welcome.

savek-cc commented 1 year ago

Looking back at the original question, this is exactly what I'm currently trying to achieve; The child relation in DT 4.7 is already a huge step - but if I understand it correctly, I currently can not use a project A as a component within another project B? I tried assigning a CPE to the Project A and create a component with the same name, version and CPE in Project B - but that doesn't seem to work -- but would be exactly what this request is about? @stevespringett about the help wanted: I guess you'd actually need a good PR and not some POC-code right?

stevespringett commented 1 year ago

@savek-cc yes, I'd need to good PR. One that did not replicate the vulnerabilities up to the parent (they will still need to be separated), but a PR that ideally aggregates them dynamically

savek-cc commented 1 year ago

@stevespringett basically I was thinking about providing each dependencytrack project an identity including a unique identifier (purl) - and the allow including an existing project into another projects as a dependency by its identifier. Basically handle DT projects like any other external 3rd party component that can be added as a dependency of another project. This way, the vulnerabilities of the (own/internal) library component would be aggregated for all of its users - just like with any other 3rd party lib that is being used (like log4j-core in a vulnerable version).

Do you have hints what might might be the issue with this way of handling - other than taking care not to create loops? (Assign own/internal purls to internal component projects in DT and then just reference these via their purl from other (consuming) projects)

roadSurfer commented 1 year ago

DT already marks components as internal (using group or name in InternalComponentIdentificationUtil, rather than the PURL or something). Would isInternal being tue not be the trigger to go and see if a unique project can be identified? If one is found, report the vuln information from that? Perhaps this could happen in ComponentQueryManager.getComponents when including metrics?

Although if you mean passing up policy/vuln info from an actual DT child project to a parent, then that'll be a bit different.

roadSurfer commented 1 year ago

I took an initial run at this. Seems to work and the code can be seen in this branch. In the example below, the "contracts" component is also a project within DT and in the second picture you can see the vulns from the porject are getting reported.

Before: Before After: After

The change starts in ComponentQueryManager.getComponents with a call to an updated getMostRecentDependencyMetrics. It works, I am not overly confident about how I am finding the project (PURL or CPE or SWID), the shallowClone... methods, ~"risk score" is missing~, I don't think this updated information is represented in the project overview, and I also think there there will need to be UI changes (e.g. "Go to project" or something similar.

Comments/Opinions welcome.

savek-cc commented 1 year ago

After looking at the code and the screenshots, that exactly the behaviour I was thinking about. I had to read up on "internal components" (https://docs.dependencytrack.org/datasources/internal-components/) as I haven't been using that one yet. If this is used as a trigger, I'd probably have to read up on regex on how to identify all of the different varieties that might end up in the system (or in the longer run make the "internal" a property that can just be set on a component). As I'm not a Java Developer by trade (and only familiar enough with the DT codebase to hack my way around - but not contribute in a meaningful way), I can't really tell what the code should look like though ;).

roadSurfer commented 1 year ago

Thinking about it more, what I wrote was at best naïve and at worst hot garbage. A project referenced as a component may not actually be the same as the project itself as dependencies can be excluded.

The scenario:

Project A depends on a vulnerable X component.
Project B depends what on A, and thus also X.
Project C also depends on A, but actively excludes X. How can we deal with this? The component A in this scenario does not have X its tree and thus does not have the same risk or vulns as project A.

Also, I don't think the risk and vulns shown in the component view are a summary of the component's dependency tree in that instance; they are just the risk and vulna of that specific component. Inherited risk/vulns are shown at the project level. So by doing what I did, I have actually started to mix slightly different datasets.

All that said, being able to match a component as a project could still be useful as component names may not match project names (given how project names have to effectively be unique).

savek-cc commented 1 year ago

I'm thinking:

"Project A, Version 1" is some sort of library (or even microservice if you want) tracked as a DT project with a list of components and their vulnerabilities as well as a summary and risk score on the project level, it might include "Component X".
"Project B" now uses this "Project A" as one of its components.
"Project C" now uses the same "Project A" - but somehow manages to exclude "Component X"?

From "Project B" point of view, it is using a component "Project A" that might or might not be made up of other components (including a "Component X"). This "Project A, Version 1" component is (probably) developed by a different team and/or under a different release schedule. So all fixing/updating has to happen in "Project A", leading to a "Project A, Version 2" which has fixed for the vulnerability (or marked them as not relevant or whatever). Once "Project B" then updates to "Project A, Version 2", they get rid of the vulnerable dependency. "Project B" really shouldn't care, what sub-component (X) of "Project A" is actually responsible for the vulnerability - because that information should be encapsulated in "Project A" itself.

So I don't think that all (or rather any) components (including "Component X") from "Project A" should even be visible at "Project B" scope - but rather allow this level of encapsulation to have "Project A" vulnerabilities to be dealt at "Project A" level. We also have external dependencies (such as a Matlab runtime) which we already manage as an "external" internal component because you won't be getting vulnerability info from the vendor - but if you use sync, you can get a somewhat "OK-ish" list of vulnerabilities within that "component" - still I would not want to have all components listed in that "Matlab-Runtime" project be mirrored into my "Project B".

Do you think that in this szenario any exclusion or not topics override the need to display vulnerabilities (summary) in common library projects on a "product-project" level?

roadSurfer commented 1 year ago

Exclusions are simple with the likes of Maven. When project C declares it depends on project A, it can also declare it is excluding component X. This is actually pretty common and means C depends on a variant of A and not the canonical A. Then, just to add confusion, we have scenarios where versions are mutable (Docker is a great example). Not sure how DT squares that circle.

As I say I don't think the components list is showing inherited risk & vulns, just ones within that component itself. Showing the project risk & vulns is thus actually showing different data. And yes, Project B would list X as it ultimately has a dependency upon it. Project C would not show X as it depends on a variant of A but if the risk & vuln of project A were shown, then people would think C is vulnerable when it is not and the information from A with X would not be represented in project C's overall data (because it exlcudes X).

I was wondering if maybe a "Directs only" view of components which collapses the list and shows inherited risk/vulns might be a way forwards.

msymons commented 1 year ago

@savek-cc

basically I was thinking about providing each dependencytrack project an identity including a unique identifier (purl) - and the allow including an existing project into another projects as a dependency by its identifier.

Why not use the purl from the BOM metadata?

eg...

    </tools>
    <component type="application" bom-ref="pkg:maven/net.foo.cas/funky-gibbon@1.6.3?type=pom">
      <group>net.foo.cas</group>
      <name>funky-gibbon</name>
      <version>1.6.3</version>
      <licenses/>
      <purl>pkg:maven/net.foo.cas/funky-gibbon@1.6.3?type=pom</purl>
      <externalReferences><reference type="build-system"><url>http://ci1-clc.bar.com:8080/job/funky-gibbon-Service-Nightly/</url></reference><reference type="distribution"><url>https://corp-nxs01.bar.com/repository/maven-releases</url></reference><reference type="issue-tracker"><url>https://jira.foo.com/projects/CAS/</url></reference><reference type="vcs"><url>https://github.com/foo/funky-gibbon-service</url></reference></externalReferences>
    </component>
  </metadata>

Doing so would (I think) tick off one of the fields that would help towards completing #1733. Which, as you might note, I re-assigned from milestone 4.8 to 4.9. But that does stop anyone from getting started 😄

savek-cc commented 1 year ago

@msymons yes - but in this case, the "pkg:maven/net.foo.cas/funky-gibbon@1.6.3" would be a component that is (vulnerability) managed within dependencytrack itself and vulnerability-info could not be obtained from NVD, Sonatype OSSI or whatever. Maybe it's not even deployed to anywhere but rather (e.g.) a C-lib (using openssl for example) that projects pull in as a DLL. I still want the "consuming" projects to be aware that/if there are vulnerabilities within this internal component (which has its own DependencyTrack project where it itself e.g. lists OpenSSL 1.1.1n as being compiled into the DLL).

nscuro commented 1 year ago

For applications composed of multiple individual modules ("backend" and "frontend" in the simplest case), having some sort of risk inheritance ("app" depends on "backend", thus it inherits vulnerabilities, policy violations etc.) is fine.

But as I think @roadSurfer is hinting at, it does not always work this way, especially when the component tracked in a different DT project is a library or framework. In many build systems, the final dependency tree is not resolved until the "top level" project is built.

If a shared library is tracked as project within DT, its dependencies (and thus vulnerabilities, policy violations, etc.) are not necessarily inherited by projects depending on that shared library. Granted, you can make this assumption with statically compiled DLLs, but definitely not with Maven packages, Go modules, Node modules or Python packages.

@msymons @savek-cc The idea of linking projects based on identifiers like PURL is good, but there are some complications that I see:

What if the PURL refers to a -SNAPSHOT version? In Maven, it is common to publish pre-releases as snapshots. This means that the contents of two artifacts could be entirely different, despite both of them claiming to be 1.0.0-SNAPSHOT for example. How do we know we're linking the correct thing?
- We could take hashes into the equation. Only perform the linking when both PURL and hashes match.
How strict would we need to be when it comes to PURL qualifiers? Repository- and download URLs can be set in a PURL, what do we do when those do not match?

rdicroce commented 1 year ago

I've opened https://github.com/DependencyTrack/dependency-track/discussions/2648 as an offshoot of this discussion. It's related but not exactly the same thing, so I didn't want to derail this, but people who've commented here might be interested in that.

As for this issue specifically, the CycloneDX spec says this about the dependencies element:

Components that do not have their own dependencies MUST be declared as empty elements within the graph. Components that are not represented in the dependency graph MAY have unknown dependencies.

So maybe part of the solution is for DT to check if a dependency entry exists in the SBOM for a given component. If a dependency entry exists, then DT assumes it's accurate and does not try to link the component to other DT projects. That should help deal with Maven-like systems where dependencies can be excluded. DT could maybe also look at the component's externalReferences and see if there's a reference to another SBOM. If there is, that seems like a reasonable sign that DT should try to link to another project.

MichaelGissingNC commented 10 months ago

I want to chime in here and offer my perspective.

The Issue

In my organization we also have the situation that we maintain certain software projects that will then be re-used by others. Let's call them plugins here.

For me this whole topic is about organizational responsibilities and accountability. From that perspective, the plugins are considered somehow monoliths. You consume them as a whole. If something doesn't work, you contact the responsible team. So also from a component analysis perspective, it's not something that I would analyze more deeply, in the SBOM I want just the reference to that plugin. The fact that this is technically just a maven dependency and that I could analyze its dependencies and also technically use maven to exclude some of the transient dependencies is not really important from that point of view. The dependency I maintain in a consuming project is the plugin - and I just want to know:

does it have known issues?
what's the risk?
is there a new version?

I can then use this information to get into the discussion with the plugin's owners about how they want to address the findings.

There's even another angle to the "technically it's a maven dependency". Some of our plugins are shipped as fat jars or even zips, so while they use maven to be built and have their own SBOM and dependency track project, the consumers won't see the maven dependencies or any transient dependencies. The maven artifact that's included in the consumer is just the fat jar. One could now argue that's bad practice, but it's a real world use-case.

Solution?

As others have stated, this is not a trivial requirement. I would offer a different view on how this could be modeled.

Let's consider a virtual Dependency Track vulnerability source/analyzer.

It could work as follows:

It can be en-/disabled and configured in the settings
For each component in a project it will look at the information of all existing projects whether it find one that matches the component's coordinates
- Here we have of course again the issue of correctly identifying something. However, if both the component and the DT project have a PURL this should be doable
- I would like to see a separation between project name in DT (should be human readable) and the artifact's technical name - but that's another discussion
If a matching component is found, a vulnerability is created for each of the vulnerabilities in the DT project, mirroring its properties
Thus, a project depending on the plugin would show this plugin as vulnerable without having to deal with the internals of the plugin. The vulnerabilities can even be audited per project.

What wouldn't work is something like "show me all projects that use log4j" when the dependency is hidden in the plugin. However, for the use-case described above, this isn't possible now anyway, at least with this proposed mechanism the plugin would light up as "carrying risk / being vulnerable".

savek-cc commented 10 months ago

"For each component in a project it will look at the information of all existing projects whether it find one that matches the component's coordinates" as you can already assign a CPE and/or PURL to a project, this is basically done.

We actually started testing creating such "reference" components - with the same PURL we assigned to the other project.

While it's currently not evaluated by DependencyTrack, we were able to do so via a script (basically: Get components for project A, search list of components for PURLs matching our naming, search for (the, one) project that has this identifier and include it's vulnerabilities in the summary calculation).

But it's not pretty and not supported by the UI. In the components-view, it could just be included like any other dependency with the list of sums of contained vulnerabilities. I'm wondering what should be shown in the "analyze vulnerabilities" section though - probably none of the inherited vulnerabilities? Or all of them?

And what's probably more important: What would the data model look like? Just "xref" component which is then dynamically populated at runtime whenever there is a query for "all vulnerabilities in project X"?

MichaelGissingNC commented 10 months ago

I'm wondering what should be shown in the "analyze vulnerabilities" section though - probably none of the inherited vulnerabilities? Or all of them?

I would say all of them - but that's something that needs discussion.

And what's probably more important: What would the data model look like? Just "xref" component which is then dynamically populated at runtime whenever there is a query for "all vulnerabilities in project X"?

Without having deep knowledge of the code, I was thinking:

Use the existing data model
In my case, the plugins are already components in the consuming projects (let's call it App)
So when App's components are analyzed for known vulnerabilities, the new analyzer would look for a matching DT project (let's call it aPlugin), then
- for each of aPlugin's vulnerabilities, create a new vulnerability in the DB - but with the new analyzer type - if it doesn't exist already
- semantically these are now vulnerabilities of aPlugin, like if somebody would have created a CVE for it
- there might be some housekeeping needed - e.g. remove a vulnerability if the original vulnerability at aPlugin is gone
- the newly created vulnerability is now assigned to the aPlugin component in the App DT project
now aPlugin's vulnerabilities are visible on App's component representing aPlugin
- the vulnerabilities can also be audited, maybe they can't be exploited in App

savek-cc commented 6 months ago

Reference: https://github.com/DependencyTrack/dependency-track/issues/2041#issuecomment-1422705865 and following comments

DependencyTrack / dependency-track