After pipx install: AttributeError: 'HTTPResponse' object has no attribute 'strict'

moi90 commented 1 year ago

Checklist

[X] I added a descriptive title
[X] I searched open reports and couldn't find a duplicate

What happened?

I installed conda-lock using pipx. After the installation, the conda-lock command fails with the following stacktrace:

$ pipx install conda-lock

$ which conda-lock
/home/.../.local/bin/conda-lock

$ conda-lock -f environment.base.yml
Locking dependencies for ['linux-64', 'osx-64', 'win-64']...
INFO:conda_lock.conda_solver:linux-64 using specs ['python 3.10.*', 'numpy 1.22.*', 'pandas', 'h5py 3.6.*', 'scikit-learn 1.0.*', 'scipy', 'pip *', 'hdbscan', 'pillow', 'pytorch', 'torchvision', 'hiredis', 'joblib 1.1.0']
Traceback (most recent call last):
  File "/home/.../.local/bin/conda-lock", line 8, in <module>
    sys.exit(main())
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 1353, in lock
    lock_func(
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 1083, in run_lock
    make_lock_files(
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 408, in make_lock_files
    lock_content = lock_content | create_lockfile_from_spec(
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 801, in create_lockfile_from_spec
    deps = _solve_for_arch(
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 737, in _solve_for_arch
    pip_deps = solve_pypi(
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/pypi_solver.py", line 269, in solve_pypi
    result = s.solve(use_latest=to_update)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/puzzle/solver.py", line 65, in solve
    packages, depths = self._solve(use_latest=use_latest)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/puzzle/solver.py", line 233, in _solve
    result = resolve_version(
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/mixology/__init__.py", line 7, in resolve_version
    return solver.solve()
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/mixology/version_solver.py", line 84, in solve
    next = self._choose_package_version()
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/mixology/version_solver.py", line 372, in _choose_package_version
    dependency = min(*unsatisfied, key=_get_min)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/mixology/version_solver.py", line 364, in _get_min
    len(self._provider.search_for(dependency)),
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/puzzle/provider.py", line 139, in search_for
    packages = self._pool.find_packages(dependency)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/repositories/pool.py", line 170, in find_packages
    packages += repo.find_packages(dependency)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/repositories/pypi_repository.py", line 101, in find_packages
    info = self.get_package_info(dependency.name)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/repositories/pypi_repository.py", line 202, in get_package_info
    return self._cache.store("packages").remember_forever(
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/cachy/repository.py", line 174, in remember_forever
    val = value(callback)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/cachy/helpers.py", line 6, in value
    return val()
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/repositories/pypi_repository.py", line 203, in <lambda>
    name, lambda: self._get_package_info(name)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/repositories/pypi_repository.py", line 207, in _get_package_info
    data = self._get("pypi/{}/json".format(name))
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/conda_lock/_vendor/poetry/repositories/pypi_repository.py", line 315, in _get
    json_response = self.session.get(self._base_url + endpoint)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/requests/sessions.py", line 745, in send
    r.content
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/requests/models.py", line 899, in content
    self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/requests/models.py", line 816, in generate
    yield from self.raw.stream(chunk_size, decode_content=True)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/urllib3/response.py", line 935, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/urllib3/response.py", line 874, in read
    data = self._raw_read(amt)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/urllib3/response.py", line 809, in _raw_read
    data = self._fp_read(amt) if not fp_closed else b""
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/urllib3/response.py", line 794, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/cachecontrol/filewrapper.py", line 96, in read
    self._close()
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/cachecontrol/filewrapper.py", line 76, in _close
    self.__callback(result)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/cachecontrol/controller.py", line 353, in cache_response
    self._cache_set(cache_url, request, response, body, expires_time)
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/cachecontrol/controller.py", line 274, in _cache_set
    self.serializer.dumps(request, response, body),
  File "/home/.../.local/pipx/venvs/conda-lock/lib/python3.10/site-packages/cachecontrol/serialize.py", line 54, in dumps
    u"strict": response.strict,
AttributeError: 'HTTPResponse' object has no attribute 'strict'

I suspect a version problem with urllib3. Recently, 2.0 was released which also caused problems elsewere, e.g. https://github.com/python-poetry/poetry/issues/7936.

$ pipx runpip conda-lock list
Package             Version
------------------- --------
appdirs             1.4.4
CacheControl        0.12.11
cachy               0.3.0
certifi             2023.5.7
cffi                1.15.1
charset-normalizer  3.1.0
click               8.1.3
click-default-group 1.2.2
clikit              0.6.2
conda_lock          1.4.0
crashtest           0.3.1
cryptography        40.0.2
distlib             0.3.6
ensureconda         1.4.3
filelock            3.12.0
html5lib            1.1
idna                3.4
importlib-metadata  6.6.0
jaraco.classes      3.2.3
jeepney             0.8.0
Jinja2              3.1.2
keyring             23.13.1
lockfile            0.12.2
MarkupSafe          2.1.2
more-itertools      9.1.0
msgpack             1.0.5
packaging           23.1
pastel              0.2.1
pip                 23.1.1
pkginfo             1.9.6
platformdirs        3.5.1
pycparser           2.21
pydantic            1.10.7
pylev               1.4.0
PyYAML              6.0
requests            2.30.0
ruamel.yaml         0.17.26
ruamel.yaml.clib    0.2.7
SecretStorage       3.3.3
setuptools          67.7.1
six                 1.16.0
tomli               2.0.1
tomlkit             0.11.8
toolz               0.12.0
typing_extensions   4.5.0
urllib3             2.0.2
virtualenv          20.23.0
webencodings        0.5.1
wheel               0.40.0
zipp                3.15.0

I see that this is already fixed in 5a33053, but it seams that this fix is not yet on PyPI.

I solved the problem doing the following:

$ pipx inject conda-lock git+https://github.com/conda/conda-lock
  injected package conda_lock into venv conda-lock
done! ✨ 🌟 ✨

$ conda-lock -f environment.base.yml
Locking dependencies for ['linux-64', 'osx-64', 'win-64']...
INFO:conda_lock.conda_solver:linux-64 using specs ['python 3.10.*', 'numpy 1.22.*', 'pandas', 'h5py 3.6.*', 'scikit-learn 1.0.*', 'scipy', 'pip *', 'hdbscan', 'pillow', 'pytorch', 'torchvision', 'hiredis', 'joblib 1.1.0']
INFO:conda_lock.conda_solver:osx-64 using specs ['python 3.10.*', 'numpy 1.22.*', 'pandas', 'h5py 3.6.*', 'scikit-learn 1.0.*', 'scipy', 'pip *', 'hdbscan', 'pillow', 'pytorch', 'torchvision', 'hiredis', 'joblib 1.1.0']
INFO:conda_lock.conda_solver:win-64 using specs ['python 3.10.*', 'numpy 1.22.*', 'pandas', 'h5py 3.6.*', 'scikit-learn 1.0.*', 'scipy', 'pip *', 'hdbscan', 'pillow', 'pytorch', 'torchvision', 'hiredis', 'joblib 1.1.0']
 - Install lock using: conda-lock install --name YOURENV conda-lock.yml

Conda Info

No response

Conda Config

No response

Conda list

No response

Additional Context

No response

maresb commented 1 year ago

Thanks for the report!

Do you install from lockfiles with conda-lock or micromamba?

Since you're currently working on the main branch, due to the bug #411, in the event that you install from lockfiles with conda-lock, then you should instead inject git+https://github.com/maresb/conda-lock@fix-411 as per #412.

mariusvniekerk commented 1 year ago

this is being caused by urllib3 >=2. If you downgrade urllib3 to pre 2.0 things should work

moi90 commented 1 year ago

Thanks, @mariusvniekerk, but we already knew this ;) (Commit 5a33053 pins urllib3 to <2.0.0)

maresb commented 1 year ago

The fix referenced above is merged into main, and I expect to cut a new release within a few days.

moi90 commented 1 year ago

Very cool, thanks!

conda / conda-lock