false positive software package vuln for Twisted web python

[pacamaster]

Fleet version: <!-- Copy this from the "My account" page in the Fleet UI, or run fleetctl --version --> 4.50.3 Web browser and operating system: Current browser and OS

---

💥  Actual behavior

False reporting for CVE-2020-10109: Twisted Web / python3-twisted .deb version is not showing vuln Pip version is

Guessing the .deb installed version twisted18.9.0-11ubuntu0.20.04.3 gets referenced against Ubuntu OVAL and has a backported patch, whereas the PIP package was referenced against NVD and is a false positive.

UPDATE (2024/08/13): Another customer example:

Hi fleet team, I'm wondering about why fleetDM is showing that python 3.8.10-0ubuntu1~20.04.9 is affected by CVE-2022-48565 and CVE-2022-48560 when the Ubuntu security notifications state that 3.8.10-0ubuntu1~20.04.9 isn't affected. Thanks for your help!

🧑‍💻  Steps to reproduce

• create ubuntu host
• apt install python3-twisted on host
• add host to Fleet
• Twisted appears 2x (deb_package && python_package)

🕯️ More info (optional)

• Python packages should only appear once for Ubuntu:
  • if installed by dpkg, the it appears as a deb_package
  • if installed by pip, the name is converted to python3- + packagename to match OVAL definitions

[JoStableford]

Related to a Slack conversation

[sharon-fdm]

Hey team! Please add your planning poker estimate with Zenhub @getvictor @lucasmrod @mostlikelee

[mostlikelee]

@pacamaster this doesn't look to be a false positive on the pip installed package

as the version number seems to be in scope. Am i missing something there? Is this a false negative on the debian installed package? Or is NVD incorrect here?

[sharon-fdm]

Hey team! Please add your planning poker estimate with Zenhub @jacobshandling @RachelElysia

[stefanamaerz]

Hi @mostlikelee !

Twisted can be installed via PIP or via a debian package.

sudo pip3 install Twisted

or

sudo apt install python-twisted

in our environment, this package is installed via a debian package:

# dpkg -L shows the files/directories installed via the python3-twisted package:
$ dpkg -L python3-twisted | grep /usr/lib/python3/dist-packages/twisted/_version.py
/usr/lib/python3/dist-packages/twisted/_version.py
$ cat /usr/lib/python3/dist-packages/twisted/_version.py

# This file is auto-generated! Do not edit!
# Use `python -m incremental.update Twisted` to change this file.

from incremental import Version

__version__ = Version('Twisted', 18, 9, 0)
__all__ = ["__version__"]

Note the version number.

I verified that this package came from an official ubuntu mirror.

https://ubuntu.com/security/CVE-2020-10109

• ubuntu's docs indicate we are not vulnerable (note the lack of an ubuntu focal entry)

https://nvd.nist.gov/vuln/detail/CVE-2020-10109

• the CPE string from NVD would indcate the 18.9.0 is vulnerable

So my understanding of the situation:

1. Canonical backports security patches to their .deb
2. the OVAL feed is correctly identifying it as not vulnerable since Canonical published a backported patch
3. Canonical does not update the pip package version for compatibility reasons (would likely break pip packaging/dependencies)
4. FleetDM is enumerating these PIP packages and deb packages twice. The .deb is correctly marked as not vulnerable, the pip enumerated version is marked as vulnerable even though it isn't.

In my mind twisted is not the bug here; its simply symptomatic of a bigger issue: ideally Fleetdm would not enumerate pip packages twice that are installed via .deb.

Let me know if that makes sense!

[mostlikelee]

@stefanamaerz I believe what we're running into here is that Fleet is scanning python packages 2x: Once by the OVAL scanner, and again by the NVD scanner.

Although we don't (yet) surface this information in Fleet, the source of any vulnerability is stored in the database in the software_cve table as an enum in the source column. We can confirm that the vulnerability is discovered in NVD by looking up the CVE and confirming source == 0 (0 for NVD, 2 for OVAL)

Do you forsee any possible issues creating false negatives by removing Ubuntu python packages from the NVD scanner?

[mostlikelee]

CC: @sharon-fdm @noahtalerman @zayhanlon

We may need a bit more time to evaluate the impact of removing Ubuntu Python Packages from the NVD scanner.

[stefanamaerz]

@mostlikelee makes sense. I wouldn't forsee removing canonical/ubuntu provided python packages from the NVD scanner as an issue; rather I would expect it to be higher fidelity given that Canonical maintains the packages and curates the OVAL feeds. If something like that is easy to do, that would be fantastic.

[noahtalerman]

@stefanamaerz sorry for the delay!

Fleet is scanning python packages 2x: Once by the OVAL scanner, and again by the NVD scanner.

@mostlikelee and @lucasmrod do y'all know why Fleet is doing this? Was it an oversight?

If y'all feel good about it, I think using OVAL only makes sense. I just want to make sure we didn't deliberately add OVAL + NVD for some reason (i.e. fixing another bug)

[lucasmrod]

Was it an oversight?

Correct. It was an oversight.

[zayhanlon]

@noahtalerman can this go back to the release board then or does it need design work from product?

[JoStableford]

Related to a Slack conversation

[noahtalerman]

Hey @zayhanlon and @sharon-fdm I think this one can go back to the release board. I moved it to the release board.

According to @lucasmrod (comment here), it sounds like there's no reason why python packages twice (once w/ OVAL and once w/ NVD).

cc @mostlikelee

[zayhanlon]

Thanks!

[sharon-fdm]

FYI @mostlikelee. We can pick one (Oval / NVD). Can you think which would be better? (Doesn't matter?)

[mostlikelee]

We should only scan in Oval. NVD should be redundant here.

[sharon-fdm]

Please add your planning poker estimate with Zenhub @iansltx

[mostlikelee]

The proposed solution here is to reclassify Ubuntu python packages to the source: python_packages_ubuntu, as well as to prefix package names with python3-{packageName} in order to correctly name match against the OVAL feed. This is going to design review tomorrow.

[noahtalerman]

Hey @mostlikelee, I just watched the Gong here. I think we want to avoid exposing a new python_packages_ubuntu source to our users if it's possible/not too difficult. This would be inconsistent w/ how software inventory works in Fleet: one osquery table = one source in software inventory.

Can this to work so that we use OVAL for Linux python packages and NVD for macOS and Windows python packages w/o adding a new source/type?

That’s what we already have documented (“Linux” column and “Packages” row): https://fleetdm.com/guides/vulnerability-processing#coverage

[mostlikelee]

@noahtalerman we'll have to differentiate between a python package on ubuntu vs the same package on another platform. If they both have the same source, then we should probably add something in the UI otherwise it could look like a bug:

[sharon-fdm]

QA DRI - @iansltx

[iansltx]

Revised QA checklist because I misunderstood desired behavior. Trying this again.

[iansltx]

To QA this, I pulled an Ubuntu 20.04 image on Docker and installed Twisted 18.9.0:

1. Globally via apt - shows up as a deb with python3-twisted and python3-twisted-bin entries at version 18.9.0-11ubuntu0.20.04.4 with no vulns
2. Globally via pip - shows up as a Python package (python3-twisted) with no vulns

Is that expected behavior, including the direct-from-pip install?

For RHEL (tested ubi8 and ubi9), e.g. python3-twisted isn't available via yum/dnf. I installed via pip and while e.g. /usr/local/lib64/python3.6/site-packages has the package in it, I can't even see twisted in the python_packages table, so for RHEL at least osquery isn'tpicking it (or some other global pip deps) up. Not sure how concerned we should be with that, given they're in the same dir as deps we do pick up?

I also tested with Debian. The only version I could pull Twisted with on .deb was via Bookworm, which let me pull 22.4.0. That got me this:

So we're seeing a similar duplicate between .deb package, which has no vulns listed, and the Python package, which has vulns: CVE-2024-41671, CVE-2023-46137, CVE-2022-39348

A forced global install via pip only shows up once, and isn't vulnerable because it's 24.7.0.

Seems like we need to clean up standard Debian as well, and potentially hunt down why osquery isn't pulling everything on the Red Hat side? These items are probably out of scope for this particular ticket, so this probably passes QA, but it feels like there's more related work to do.

[iansltx]

Looks like the 18.9.0 Twisted behavior via pip on Ubuntu matches 4.56.0, so this isn't a regression and is probably fine.

Also, it feels like there's another issue embedded in this one: python 3.8.10-0ubuntu1~20.04.9 itself being a false-positive? Seems like that's a completely different issue, and not covered by the work done here?

[mostlikelee]

Looks like expected behavior to me, as we kept the scope down to just Ubuntu in case there are unexpected effects of the change. Agree the RHEL issue and the python false positive issues are unrelated.

[iansltx]

Okay. QA passed then.

[fleet-release]

False positive fixed, Twisted web python secured, Fleet's trust restored, pure.