Search Space produces better results with package.name checks

[torque59]

The underlying issue seems to still not be solved with adding more package specific names https://github.com/AppThreat/dep-scan/issues/23.

When adding new packages to requirements.txt, this seems to fail. As an example, scan now fails to detect the following packages as an example:

tensorflow
ansible

Searching in terms of package name seems to produce better results at least for requirements.txt. This shouldn't break other language dependencies in lock files ideally.

[prabhu]

Hi @torque59,

Thanks for this PR. Unfortunately, removing vendor would lead to large number of false positives since the same package name might appear under different languages. The caller of vdb can pass None for vendor to force a package name based lookup.

So dep-scan for example could create a variation for a package with vendor as None which should either work or we could make it work.

[prabhu]

This is handled in depscan as part of https://github.com/AppThreat/dep-scan/commit/3c0123a1ab86a06f7b109ae4482bc6dbdf3c3306

https://github.com/AppThreat/dep-scan/blob/master/depscan/lib/normalize.py#L79

[prabhu]

Also note that NVD refers to tensorflow package as:

• tensorflow
• tensorflow-cpu
• tensorflow-gpu

and so on. The package name really depends on the mood of the person filing the issue :)

[prabhu]

@torque59 Did you get a chance to test with the latest version of scan?

[torque59]

@prabhu yes the latest version looks good. But i did see the commit, would the whitelist sort approach work in the long run, like we would still have to keep this extending the list right for new vendors ?

[prabhu]

@torque59 for python, it is currently doing both vendor alias search and package-based search. So, there might be new false positives which might require tuning at some point, but false negatives should be reduced to an extent.

It's a known industry-wide problem. Large companies have resorted to human curation and updates for vulnerabilities or licensing the dataset from commercial providers. The resulting database unfortunately is not free and comes with restrictive licensing clauses and so on. In the past, I have tried looking into the description text as well started a project https://github.com/AppThreat/cve-annotations to improve the NVD database.

[prabhu]

@torque59 can this be closed. Are you happy with the fixes made to depscan?

[torque59]

@prabhu yes this can be closed, apologies for leaving it open. The fixes look good now with some of the packages i tested. I'll open up issues incase i run into any of them. Thank you @prabhu for the effort :)