Closed r-findley closed 3 weeks ago
Thanks for the report @r-findley!
Just to make sure I understand: what you're seeing here is idna==3.6
when run via the pre-commit
hook, but not when auditing the environment directly, right?
Are you able to share more about your dependency/requirements set? I'm curious if idna==3.6
is being selected during initial resolution due to an upper bound (or aggressive caching) somewhere.
Hello @woodruffw! I appreciate the quick reply. I'll paste my requirements.txt info below. You are correct though. When I ran the pre-commit hooks during the commit, I got the warning about idna being vulnerable at 3.6. I did an update to idna to the 3.7 but never actually updated my requirements.txt file. You'll see below this shows idna at 3.7.
azure-functions 1.18.0
bandit 1.7.8
black 24.3.0
boolean.py 4.0
CacheControl 0.14.0
certifi 2024.2.2
cfgv 3.4.0
charset-normalizer 3.3.2
click 8.1.7
cyclonedx-python-lib 6.4.4
defusedxml 0.7.1
distlib 0.3.8
et-xmlfile 1.1.0
filelock 3.13.1
html5lib 1.1
identify 2.5.35
idna 3.7
iniconfig 2.0.0
isort 5.13.2
license-expression 30.3.0
markdown-it-py 3.0.0
mdurl 0.1.2
msgpack 1.0.8
mypy-extensions 1.0.0
nodeenv 1.8.0
numpy 1.26.4
openpyxl 3.1.2
packageurl-python 0.15.0
packaging 24.0
pandas 2.2.1
pathspec 0.12.1
pbr 6.0.0
pip 23.3.1
pip-api 0.0.33
pip_audit 2.7.2
pip-requirements-parser 32.0.1
platformdirs 4.2.0
pluggy 1.4.0
pre-commit 3.6.2
py-serializable 1.0.2
Pygments 2.17.2
pyparsing 3.1.2
pytest 8.1.1
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
pytz 2024.1
PyYAML 6.0.1
requests 2.31.0
rich 13.7.1
setuptools 69.0.2
six 1.16.0
sortedcontainers 2.4.0
stevedore 5.2.0
toml 0.10.2
tzdata 2024.1
urllib3 2.2.1
virtualenv 20.25.1
webencodings 0.5.1
Even after performing the install of 3.7, the pre-commit is failing. But a run of pip-audit in the terminal is working fine and not flagging any vulnerabilities in idna 3.7.
Got it, thank you!
Hmm, that's odd. Could you share your pre-commit
config for pip-audit
? I'm curious if you're currently configuring it to run on the requirements.txt
(or similar) rather than on your environment.
(FWIW, I know relatively little about pre-commit
, but I've seen similar things before where people run pip-audit
and get different results because some invocations run it against their requirements input versus their current environment π
)
Here's my .pre-commit-config.yaml. I commented out the pip-audit one since it was not working correctly. One note - I just removed the flag for the requirements.txt because my coworker on a Windows machine was running into an issue where the pre-commit would hang indefinitely if he used the actual requirements file. When I was running it, it failed either way.
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
- id: check-json
- id: detect-private-key
- id: flake8
args: ["--max-line-length=88", "--ignore=E203"]
- id: no-commit-to-branch
args: ['--branch', 'master', '--branch', 'development']
- id: requirements-txt-fixer
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 22.10.0
hooks:
- id: black
- repo: https://github.com/PyCQA/bandit
rev: 1.7.5
hooks:
- id: bandit
args: ["-c", "bandit.yaml", "-r", "."]
# - repo: https://github.com/pypa/pip-audit
# rev: v2.7.2
# hooks:
# - id: pip-audit
# - repo: local
# hooks:
# - id: pytest
# name: pytest
# entry: pytest
# language: system
# pass_filenames: false
# always_run: true
This is where my knowledge of pre-commit
is lacking, but I suspect that what's happening here is that pre-commit
is actually running within its own environment, and so running pip-audit
without any arguments is really just auditing the pre-commit
environment rather than your intended development/deployment environment.
I'm not sure how to easily verify this, though π
TL;DR: I think that, unfortunately, you need to use -r requirements.txt
(or .
for pyproject.toml
mode) in pre-commit
. With that, it's insufficient to just update your dev environment with pip update
or similar -- the requirements.txt
always needs to be kept up-to-date in order for pre-commit
to not flag any findings within it.
(I'm curious to hear more about the hang with Windows as well -- that sounds a lot like #756, which hasn't made it into a release yet.)
Ah you may be right. We had removed the -r requirements.txt
call due to the Windows issue (It may be #756 though I can't recall exactly if that's what my co-worker mentioned). I added the flag back to my recent commit and it worked properly. I believe that means it was an issue on our end though I'll keep an eye on my work and the team's to see if we run into the issue after #756 is addressed.
Sounds good! I'll try and make a release containing #756 today.
The fix in #756 has been released! Can you give the latest version of pip-audit
a try and see if it resolves the hang you've seen?
No response since prospective fix, so closing. Please let me know if you're still seeing this and I'll re-open!
Pre-submission checks
Expected behavior
I expected pip-audit to pass the checks when using the pre-commit hook in my repository.
Actual behavior
After getting the following failure:
I performed an upgrade to the idna library to version 3.7 and ran the commit again. This still failed with the above error. I therefore ran the command
directly in the virtual environment and the pip-audit passed successfully.
Reproduction steps
Logs
No response
Additional context
I am running this on a MacBook Pro using Sonoma 14.4.1 while a co-worker is using Windows 11 OS Build 22631.3447
OS name, version, and architecture
No response
pip-audit version
2.7.2
pip version
23.3.1
Python version
3.11.7