txoof
commented
1 year ago Here is the full --support output. The output exceeded Github's maximum comment length.
2023.6.23.txt
2022.11.11.txt
Closed txoof closed 1 year ago
txoof
commented
1 year ago Here is the full --support output. The output exceeded Github's maximum comment length.
2023.6.23.txt
2022.11.11.txt
matteius
commented
1 year ago @txoof I wasn't able to detect an issue:
matteius@matteius-VirtualBox:~/pipenv-triage$ mkdir issue-5756
matteius@matteius-VirtualBox:~/pipenv-triage$ cd issue-5756/
matteius@matteius-VirtualBox:~/pipenv-triage/issue-5756$ ls
matteius@matteius-VirtualBox:~/pipenv-triage/issue-5756$ vim Pipfile
matteius@matteius-VirtualBox:~/pipenv-triage/issue-5756$ pipenv lock
Creating a virtualenv for this project...
Pipfile: /home/matteius/pipenv-triage/issue-5756/Pipfile
Using /home/matteius/.pyenv/versions/3.9.16/bin/python3.9 (3.9.16) to create virtualenv...
⠇ Creating virtual environment...created virtual environment CPython3.9.16.final.0-64 in 533ms
creator CPython3Posix(dest=/home/matteius/.virtualenvs/issue-5756-l4hhTZxB, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/matteius/.local/share/virtualenv)
added seed packages: pip==23.1.2, setuptools==67.8.0, wheel==0.40.0
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /home/matteius/.virtualenvs/issue-5756-l4hhTZxB
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Updated Pipfile.lock (50541ebd1e358e19fd1abf7ff833c7dbd5e3484685f8b85c606d51ffe7892041)!
matteius@matteius-VirtualBox:~/pipenv-triage/issue-5756$ pipenv sync
Installing dependencies from Pipfile.lock (892041)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!
matteius@matteius-VirtualBox:~/pipenv-triage/issue-5756$ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
pillow = "*"
[dev-packages]
[requires]
python_version = "3.9"
matteius@matteius-VirtualBox:~/pipenv-triage/issue-5756$ pipenv run pip freeze
Pillow==9.5.0note: I remove the python_full_version because I have a newer version of 3.9 available ... tried it again with both directives just in case and it installed 3.9.2 using pyenv just to be sure and that appeared to work as well.
txoof
commented
1 year ago @matteius What OS/Hardware are you running this on? I have two identical Pis running Raspbian Bullseye32; the 2022 version works great, the 2023 fails miserably in all cases that involve building wheels.
I thought maybe I had dorked up the python environment pi with 2023 so I did a fresh install and update and the problem persisted.
Any suggestions?
matteius
commented
1 year ago Ah I see, you are on a pi -- I was using an ubuntu VM -- I think the problem is actually you need your default index to not be pypi, but to be https://www.piwheels.org/simple/ -- Try swapping that out in your default source and relock/sync.
txoof
commented
1 year ago your default index to not be pypi, but to be https://www.piwheels.org/simple/
Huzzah! That worked for the trivial example. I'll do some more testing to see if things hold together for the more complex projects.
Is this in the documentation anywhere? I've scoured the intertubes, but come up pretty empty on this particular issue.
matteius
commented
1 year ago @txoof Well basically the old version you were on was scanning multiple indexes which was suspectable for package confusion attacks, which is something I've worked to prevent in pipenv, it's been a few iterations of locking down the index lookups using what I termed "Index restricted packages". The idea is that the default index is the only index to look in for packages unless a secondary index is explicitly specified, which today has to be done via the Pipfile.
matteius
commented
1 year ago More information on specifying package indexes here: https://pipenv.pypa.io/en/latest/indexes/
txoof
commented
1 year ago @matteius Thanks for the link to the docs.
Is there any way to specify the source from a requirements file? Right now I have a flat requirements.txt file.
The link to "importing from requirements" is broken.
txoof
commented
1 year ago I tried to see if I could get pipenv to generate a requirements file based on a Pipfile that contained multiple sources, but it just defaulted to the first source
requirements.txt
-i https://pypi.org/simple$
argconfigparse==0.2.7$
certifi==2022.12.7 ; python_version >= '3.6'$
charset-normalizer==3.1.0 ; python_full_version >= '3.7.0'$
docopt==0.6.2$
epdlib==0.6.1.1$
idna==3.4 ; python_version >= '3.5'$
-e git+https://github.com/GregDMeyer/IT8951.git@67215164a7fc471bc6904f72ad55e51030905a97#egg=it8951$
pillow==9.4.0 ; python_version >= '3.7'$
pipreqs==0.4.11$
requests==2.28.2 ; python_version >= '3.7' and python_version < '4'$
rpi.gpio==0.7.1$
spidev==3.6$
urllib3==1.26.15 ; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'$
yarg==0.1.9It looks like I'm stuck in dependency hell. Some of the libraries I depend worked fine through PyPi, but are broken at piwheels. Take a look at epdlib on piwheels; the ancient old versions look OK, but the more recent ones have some sort of build failure.
When I try to install epdlib, I can get an ancient version from piwheels to install, but the newer versions are all broken. If I try to install from PyPi, I get errors similar to those I shared originally.
txoof
commented
1 year ago I've sorted this out well enough to move on.
thanks for the guidance. I'll open a new issue for the documentation issue I found above.
[x] Be sure to check the existing issues (both open and closed!), and make sure you are running the latest version of Pipenv.
[x] Check the diagnose documentation for common issues before posting! We may close your issue if it is very similar to one of them. Please be considerate, or be on your way.
NOTE: Due to limitations in github comments, I had to
**snip**the--supportoutput to get it to fit. I also attached the full output as files in a comment below.Issue description
Pipenv does not appear to find local libraries/headers when building packages. Installing any module that requires zlib or other headers fails durring the install. Installing the same package using an older version of pipenv or in a simple venv (without using pipenv) succeeds. The zlib headers are installed via the debian packages: zlib1g-dev (1:1.2.11.dfsg-2+deb11u2).
Debugging experience
What I've tried to narrow down the problem:
Conclusion: My best guess is that 2023.6 is failing to find the local libraries/headers for some reason.
Details: Pipenv version 2022.11.11 successfully installs the same packages on the same hardware/python/os version. See below for the debug output from the working version (2022.11.11) and the non-working version (2023.6.23).
Installing Pillow in a simple virtual environment also works as expected.
mkdir venv_debug; cd venv_debugpython3 -m venv ./source ./bin/activatepip3 install pillowExpected result
PIpenv should fetch and install the packages without error.
Actual result
running `pipenv install Pillow --verbose` results in the following error:
``` Installing Pillow... Resolving Pillow... Adding Pillow to Pipfile's [packages] ... ✔ Installation Succeeded Installing dependencies from Pipfile.lock (ee887c)... Writing supplied requirement line to temporary file: 'pillow==9.5.0 -- hash=sha256:07999f5834bdc404c442146942a2ecadd1cb6292f5229f4ed3b31e0a108746b1 -- ** snipped due to comment length maximum ** hash=sha256:0852ddb76d85f127c135b6dd1f0bb88dbb9ee990d2cd9aa9e28526c93e794fba -- hash=sha256:fed1e1cf6a42577953abbe8e6cf2fe2f566daebde7c34724ec8803c4c0cda579' Install Phase: Standard Requirements Preparing Installation of 'pillow' $ /home/pi/.local/share/virtualenvs/PaperPi-VBShxqF-/bin/python /home/pi/.local/lib/python3.9/site-packages/pipenv/patched/pip/__pip-runner__.py install -i https://pypi.org/simple --no-input --upgrade --no-deps --exists-action=i -r /tmp/pipenv-tdqmf2lb-requirements/pipenv-ucpn9w7f-hashed-reqs.txt Using source directory: '/home/pi/.local/share/virtualenvs/PaperPi-VBShxqF-/src' Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Collecting pillow==9.5.0 (from -r /tmp/pipenv-tdqmf2lb-requirements/pipenv-ucpn9w7f-hashed-reqs.txt (line 1)) Using cached Pillow-9.5.0.tar.gz (50.5 MB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Building wheels for collected packages: pillow Building wheel for pillow (setup.py): started Building wheel for pillow (setup.py): finished with status 'error' Running setup.py clean for pillow Failed to build pillow error: subprocess-exited-with-error × python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [191 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-aarch64-cpython-39 ** snipped ** copying src/PIL/GbrImagePlugin.py -> build/lib.linux-aarch64-cpython-39/PIL running egg_info writing src/Pillow.egg-info/PKG-INFO writing dependency_links to src/Pillow.egg-info/dependency_links.txt writing requirements to src/Pillow.egg-info/requires.txt writing top-level names to src/Pillow.egg-info/top_level.txt reading manifest file 'src/Pillow.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '*.c' warning: no files found matching '*.h' warning: no files found matching '*.sh' warning: no files found matching '*.txt' warning: no previously-included files found matching '.appveyor.yml' warning: no previously-included files found matching '.clang-format' warning: no previously-included files found matching '.coveragerc' warning: no previously-included files found matching '.editorconfig' warning: no previously-included files found matching '.readthedocs.yml' warning: no previously-included files found matching 'codecov.yml' warning: no previously-included files found matching 'renovate.json' warning: no previously-included files matching '.git*' found anywhere in distribution warning: no previously-included files matching '*.pyc' found anywhere in distribution warning: no previously-included files matching '*.so' found anywhere in distribution no previously-included directories found matching '.ci' adding license file 'LICENSE' writing manifest file 'src/Pillow.egg-info/SOURCES.txt' warning: build_py: byte-compiling is disabled, skipping. running build_ext The headers or library files could not be found for zlib, a required dependency when compiling Pillow from source. Please see the install instructions at: https://pillow.readthedocs.io/en/latest/installation.html Traceback (most recent call last): File "/tmp/pip-install-2oda4kx9/pillow_3ab2bc9e4599449e8466229ad72764cc/setup.py", line 993, inSteps to replicate
mkdir env_test; cd env_testpipenv --python 3pipenv install PillowProvide the steps to replicate (which usually at least includes the commands and the Pipfile).
Pipfile:
Non-working version
$ pipenv --support
Pipenv version: `'2023.6.26'` Pipenv location: `'/home/pi/.local/lib/python3.9/site-packages/pipenv'` Python location: `'/usr/bin/python3'` OS Name: `'posix'` User pip version: `'20.3.4'` user Python installations found: PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.9.2', 'os_name': 'posix', 'platform_machine': 'aarch64', 'platform_python_implementation': 'CPython', 'platform_release': '6.1.21-v8+', 'platform_system': 'Linux', 'platform_version': '#1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023', 'python_full_version': '3.9.2', 'python_version': '3.9', 'sys_platform': 'linux'} ``` System environment variables: - `HOME` - `LANG` - `LC_TERMINAL` - `LC_TERMINAL_VERSION` - `LOGNAME` - `MOTD_SHOWN` - `OLDPWD` - `PATH` - `PWD` - `SHELL` - `SHLVL` - `SSH_CLIENT` - `SSH_CONNECTION` - `SSH_TTY` - `TERM` - `TMUX` - `TMUX_PANE` - `USER` - `XDG_RUNTIME_DIR` - `XDG_SESSION_CLASS` - `XDG_SESSION_ID` - `XDG_SESSION_TYPE` - `_` - `ZSH` - `PAGER` - `LESS` - `LSCOLORS` - `LS_COLORS` - `EDITOR` - `PYTHONDONTWRITEBYTECODE` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/usr/local/bin:/usr/bin:/bin:/usr/games:/home/pi/bin/:/home/pi/bin/develtools:/home/pi/.local/bin:/usr/local/bin` - `SHELL`: `/usr/bin/zsh` - `EDITOR`: `vim` - `LANG`: `en_US.UTF-8` - `PWD`: `/home/pi/src/env_test` --------------------------- Contents of `Pipfile` ('/home/pi/src/env_test/Pipfile'): ```toml [[source]] url = "https://pypi.org/simple" verify_ssl = true name = "pypi" [packages] pillow = "*" [dev-packages] [requires] python_version = "3.9" python_full_version = "3.9.2" ``` Contents of `Pipfile.lock` ('/home/pi/src/env_test/Pipfile.lock'): ```json { "_meta": { "hash": { "sha256": "350a7d449eeddb194fc7f106f863e61684ad7b256093e284467bbd2c45ee887c" }, "pipfile-spec": 6, "requires": { "python_full_version": "3.9.2", "python_version": "3.9" }, "sources": [ { "name": "pypi", "url": "https://pypi.org/simple", "verify_ssl": true } ] }, "default": { "pillow": { "hashes": [ **snipped due to comment maximum** ], "index": "pypi", "version": "==9.5.0" } }, "develop": {} } ```Working Version
$ pipenv --support
Pipenv version: `'2022.11.11'` Pipenv location: `'/home/pi/.local/lib/python3.9/site-packages/pipenv'` Python location: `'/usr/bin/python3'` OS Name: `'posix'` User pip version: `'22.3'` user Python installations found: - `3.9.2`: `/usr/bin/python3.9` - `3.9.2`: `/usr/bin/python3` - `3.9.2`: `/bin/python3.9` - `3.9.2`: `/bin/python3` - `2.7.18`: `/usr/bin/python2.7` - `2.7.18`: `/usr/bin/python` - `2.7.18`: `/usr/bin/python2` - `2.7.18`: `/bin/python2.7` - `2.7.18`: `/bin/python` - `2.7.18`: `/bin/python2` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.9.2', 'os_name': 'posix', 'platform_machine': 'aarch64', 'platform_python_implementation': 'CPython', 'platform_release': '6.1.21-v8+', 'platform_system': 'Linux', 'platform_version': '#1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023', 'python_full_version': '3.9.2', 'python_version': '3.9', 'sys_platform': 'linux'} ``` System environment variables: - `HOME` - `LANG` - `LC_TERMINAL` - `LC_TERMINAL_VERSION` - `LOGNAME` - `MOTD_SHOWN` - `OLDPWD` - `PATH` - `PWD` - `SHELL` - `SHLVL` - `SSH_CLIENT` - `SSH_CONNECTION` - `SSH_TTY` - `TERM` - `TMUX` - `TMUX_PANE` - `USER` - `XDG_RUNTIME_DIR` - `XDG_SESSION_CLASS` - `XDG_SESSION_ID` - `XDG_SESSION_TYPE` - `_` - `ZSH` - `PAGER` - `LESS` - `LSCOLORS` - `LS_COLORS` - `EDITOR` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PIP_PYTHON_PATH` - `PYTHONDONTWRITEBYTECODE` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/pi/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/games:/home/pi/bin/:/home/pi/bin/develtools:/home/pi/.local/bin:/usr/local/bin` - `SHELL`: `/usr/bin/zsh` - `EDITOR`: `vim` - `LANG`: `en_US.UTF-8` - `PWD`: `/home/pi/src/zlib_test` --------------------------- Contents of `Pipfile` ('/home/pi/src/zlib_test/Pipfile'): ```toml [[source]] url = "https://pypi.org/simple" verify_ssl = true name = "pypi" [packages] pillow = "*" [dev-packages] [requires] python_version = "3.9" python_full_version = "3.9.2" ```