search

intel / cve-bin-tool

The CVE Binary Tool helps you determine if your system includes known vulnerabilities. You can scan binaries for over 200 common, vulnerable components (openssl, libpng, libxml2, expat and others), or if you know the components used, you can get a list of known vulnerabilities associated with an SBOM or a list of components and versions.
https://cve-bin-tool.readthedocs.io/en/latest/
GNU General Public License v3.0
1.23k stars 463 forks source link

Discussion: OSSF Scorecard recommendation: pinning pip & other dependencies? #2673

Closed terriko closed 6 months ago

terriko commented 1 year ago

I've got a few errors from OSSF scorecard complaining that we run commands like python -m pip install --upgrade pip

OSSF wants us to pin versions as best practice. We do this elsewhere for improved developer experience with things like linters (so what pre-commit uses locally will generally match what we use in Actions) but we avoid doing it in other places (such as our own requirements.txt files) to avoid leaving users waiting for an update from us before a vulnerable component will get upgraded. (As we only release 2x per year, that could be a long wait!)

The pip-in-CI thing falls sort of halfway between those two -- we absolutely could pin our Actions components, at the cost of some extra maintenance on my end. But... does it actually provide additional security, reliability or usability for developers?

I actually do want to test that you can pip install cve-bin-tool with the latest pip, so I'm absolutely getting value from the current setup in some places, but it's possible it matters less for CI workflows where we're not actually testing our own code.

I'm leaning towards "pinning pip doesn't provide sufficient improvements in security to justify the maintenance cost" but I wanted to hear from others before I unilaterially make that decision.

Anyone got any thoughts or opinions? Are any of our other components worth switching to a pin-and-maintain model? (Maybe the HTML tests that are flaky could be improved this way?)

anthonyharrison commented 1 year ago

@terriko I think we must use the latest version which is what we get on a 'clean' install. This ensures we get what most users will experience. We could actually recommend that when users do an install of cve-bin-tool, they always use --force-reinstall to get the latest versions of all components.

Having fixed versions is good if there is a need for a stable baseline is required and I wonder if we should create a FIXED version of components for each release.

terriko commented 1 year ago

I should note that one of the reasons people recommend pinning version numbers is that it makes it easier to get cve scans done. This is true with python dependencies + dependabot as a workflow, and OSSF Scorecard definitely did check that we had dependabot enabled. We do scans outside of dependabot and can handle a "scan latest" workflow, so pinning has less value to us than it might for others using the scorecard.

Given that most of our warnings for scorecard are about pinning right now, I'm going to go ahead and mass-dismiss them so I can work on the others more easily and won't miss it if a new issue comes up in that tab, but I'll leave this issue open for further discussion and refinement. (It should be easy to un-dismiss the issues if we want to change our minds later.)

terriko commented 6 months ago

I think we've come to the conclusion that pinning pip dependencies is not for us, and we've improved our own scanning to support that. I don't particularly believe that pinning is going to yield improved security, so I'm going to go ahead and close this and we can revisit if we change our minds.