search

pypa / pipenv

Python Development Workflow for Humans.
https://pipenv.pypa.io
MIT License
24.81k stars 1.87k forks source link

I think that pipenv install tries to update all packages #4151

Closed revolter closed 2 years ago

revolter commented 4 years ago

Be sure to check the existing issues (both open and closed!), and make sure you are running the latest version of Pipenv.

Check the diagnose documentation for common issues before posting! We may close your issue if it is very similar to one of them. Please be considerate, or be on your way.

Make sure to mention your debugging experience if the documented solution failed.

Issue description

Having this commit and running pipenv install fails with:

Warning: Your Pipfile requires python_version 3.6, but you are using 3.7.6 (/Users/revolt/.local/share/v/f/bin/python).
  $ pipenv --rm and rebuilding the virtual environment may resolve the issue.
  $ pipenv check will surely fail.
Installing dependencies from Pipfile.lock (72362a)…
An error occurred while installing markupsafe==1.0 --hash=sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 14/14 — 00:00:01
Installing initially failed dependencies…
[pipenv.exceptions.InstallError]:   File "/usr/local/Cellar/pipenv/2018.11.26_3/libexec/lib/python3.8/site-packages/pipenv/core.py", line 1863, in do_install
[pipenv.exceptions.InstallError]:       do_init(
[pipenv.exceptions.InstallError]:   File "/usr/local/Cellar/pipenv/2018.11.26_3/libexec/lib/python3.8/site-packages/pipenv/core.py", line 1246, in do_init
[pipenv.exceptions.InstallError]:       do_install_dependencies(
[pipenv.exceptions.InstallError]:   File "/usr/local/Cellar/pipenv/2018.11.26_3/libexec/lib/python3.8/site-packages/pipenv/core.py", line 858, in do_install_dependencies
[pipenv.exceptions.InstallError]:       batch_install(
[pipenv.exceptions.InstallError]:   File "/usr/local/Cellar/pipenv/2018.11.26_3/libexec/lib/python3.8/site-packages/pipenv/core.py", line 763, in batch_install
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
[pipenv.exceptions.InstallError]:   File "/usr/local/Cellar/pipenv/2018.11.26_3/libexec/lib/python3.8/site-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Looking in indexes: https://pypi.python.org/simple', 'Collecting markupsafe==1.0', '  Using cached MarkupSafe-1.0.tar.gz (14 kB)']
[pipenv.exceptions.InstallError]: ['ERROR: Command errored out with exit status 1:', '     command: /Users/revolt/.local/share/virtualenvs/fastlane-OOwkva-u/bin/python3.7 -c \'import sys, setuptools, tokenize; sys.argv[0] = \'"\'"\'/private/var/folders/88/6r327zlj6bqf5t_xfzm495h00000gn/T/pip-install-fhqy6jyq/markupsafe/setup.py\'"\'"\'; __file__=\'"\'"\'/private/var/folders/88/6r327zlj6bqf5t_xfzm495h00000gn/T/pip-install-fhqy6jyq/markupsafe/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' egg_info --egg-base /private/var/folders/88/6r327zlj6bqf5t_xfzm495h00000gn/T/pip-install-fhqy6jyq/markupsafe/pip-egg-info', '         cwd: /private/var/folders/88/6r327zlj6bqf5t_xfzm495h00000gn/T/pip-install-fhqy6jyq/markupsafe/', '    Complete output (5 lines):', '    Traceback (most recent call last):', '      File "<string>", line 1, in <module>', '      File "/private/var/folders/88/6r327zlj6bqf5t_xfzm495h00000gn/T/pip-install-fhqy6jyq/markupsafe/setup.py", line 6, in <module>', '        from setuptools import setup, Extension, Feature', "    ImportError: cannot import name 'Feature' from 'setuptools' (/Users/revolt/.local/share/virtualenvs/fastlane-OOwkva-u/lib/python3.7/site-packages/setuptools/__init__.py)", '    ----------------------------------------', 'ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.']
ERROR: ERROR: Package installation failed...

After trying to find out the cause using pipenv lock --clear --verbose, I "fixed" it like so: https://github.com/fastlane/docs/pull/911/files.

I said "fixed" because those version were already present in the lock file, so it looks like pipenv was somehow ignoring them.

And what made me think it was actually trying to update all of the packages was that, after specifying those version in the Pipfile and running pipenv install updated all the packages, even though I didn't run any update command (nor pipenv update, nor pipenv update <package_name>).

I couldn't find any related issue.

$ pipenv --support Pipenv version: `'2018.11.26'` Pipenv location: `'/usr/local/Cellar/pipenv/2018.11.26_3/libexec/lib/python3.8/site-packages/pipenv'` Python location: `'/usr/local/Cellar/pipenv/2018.11.26_3/libexec/bin/python3.8'` Python installations found: - `3.7.6`: `/Users/revolt/.local/share/virtualenvs/fastlane-OOwkva-u/bin/python3` - `3.7.6`: `/usr/local/bin/python3` - `3.7.6`: `/usr/local/bin/python3.7m` - `3.7.3`: `/usr/bin/python3` - `2.7.16`: `/usr/bin/python` - `2.7.16`: `/usr/bin/pythonw` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.8.1', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '19.2.0', 'platform_system': 'Darwin', 'platform_version': 'Darwin Kernel Version 19.2.0: Sat Nov 9 03:47:04 PST ' '2019; root:xnu-6153.61.1~20/RELEASE_X86_64', 'python_full_version': '3.8.1', 'python_version': '3.8', 'sys_platform': 'darwin'} ``` System environment variables: - `PATH` - `TERM_PROGRAM` - `GEM_HOME` - `PIP_PYTHON_PATH` - `SHELL` - `TERM` - `TMPDIR` - `TERM_PROGRAM_VERSION` - `TERM_SESSION_ID` - `ZSH` - `LC_ALL` - `CDPATH` - `USER` - `NVM_DIR` - `LS_COLORS` - `COMMAND_MODE` - `SSH_AUTH_SOCK` - `__CF_USER_TEXT_ENCODING` - `VIRTUAL_ENV` - `PAGER` - `LSCOLORS` - `PIPENV_ACTIVE` - `_` - `LaunchInstanceID` - `EXA_COLORS` - `PWD` - `EDITOR` - `LANG` - `ITERM_PROFILE` - `XPC_FLAGS` - `PYTHONDONTWRITEBYTECODE` - `RBENV_SHELL` - `XPC_SERVICE_NAME` - `COLORFGBG` - `HOME` - `SHLVL` - `LC_TERMINAL_VERSION` - `ITERM_SESSION_ID` - `LOGNAME` - `LESS` - `PIP_DISABLE_PIP_VERSION_CHECK` - `LC_CTYPE` - `LC_TERMINAL` - `SQLITE_EXEMPT_PATH_FROM_VNODE_GUARDS` - `SECURITYSESSIONID` - `COLORTERM` - `PIP_SHIMS_BASE_MODULE` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: - `PIPENV_ACTIVE`: `1` Debug–specific environment variables: - `PATH`: `/usr/local/Cellar/pipenv/2018.11.26_3/libexec/tools:/Users/revolt/.local/share/virtualenvs/fastlane-OOwkva-u/bin:/Users/revolt/.rbenv/shims:/Users/revolt/.gems/bin:/usr/local/Cellar/pipenv/2018.11.26_3/libexec/tools:/Users/revolt/.rbenv/shims:/Users/revolt/.gems/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin:/Users/revolt/.local:/Users/revolt/.local/bin:/Users/revolt/.local/git-scripts:/usr/local/opt/qt/bin:/Users/revolt/Library/Python/3.7/bin:/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin:/Users/revolt/.local:/Users/revolt/.local/bin:/Users/revolt/.local/git-scripts:/usr/local/opt/qt/bin:/Users/revolt/Library/Python/3.7/bin` - `SHELL`: `/bin/zsh` - `EDITOR`: `nvim` - `LANG`: `en_US.UTF-8` - `PWD`: `/Users/revolt/Development/GitHub/docs/fastlane` - `VIRTUAL_ENV`: `/Users/revolt/.local/share/virtualenvs/fastlane-OOwkva-u` --------------------------- Contents of `Pipfile` ('/Users/revolt/Development/GitHub/docs/fastlane/Pipfile'): ```toml [[source]] url = "https://pypi.python.org/simple" verify_ssl = true name = "pypi" [packages] markdown-include = "==0.5.1" mdx-truly-sane-lists = "==1.0" mkdocs = "==0.15.3" pymdown-extensions = "==1.1" [dev-packages] [requires] python_version = "3.6" ``` Contents of `Pipfile.lock` ('/Users/revolt/Development/GitHub/docs/fastlane/Pipfile.lock'): ```json { "_meta": { "hash": { "sha256": "4caae13584446d87f4e0c52db9c23df9256afd205a7d57afd9cca94da472362a" }, "pipfile-spec": 6, "requires": { "python_version": "3.6" }, "sources": [ { "name": "pypi", "url": "https://pypi.python.org/simple", "verify_ssl": true } ] }, "default": { "click": { "hashes": [ "sha256:29f99fc6125fbc931b758dc053b3114e55c77a6e4c6c3a2674a2dc986016381d", "sha256:f15516df478d5a56180fbf80e68f206010e6d160fc39fa508b65e035fd75130b" ], "version": "==6.7" }, "jinja2": { "hashes": [ "sha256:74c935a1b8bb9a3947c50a54766a969d4846290e1e788ea44c1392163723c3bd", "sha256:f84be1bb0040caca4cea721fcbbbbd61f9be9464ca236387158b0feea01914a4" ], "version": "==2.10" }, "livereload": { "hashes": [ "sha256:422de10d7ea9467a1ba27cbaffa84c74b809d96fb1598d9de4b9b676adf35e2c", "sha256:5ed6506f5d526ee712da9f3739c27714e6f3376f3e481728d298efceae0ec83a" ], "version": "==2.5.1" }, "markdown": { "hashes": [ "sha256:9ba587db9daee7ec761cfc656272be6aabe2ed300fece21208e4aab2e457bc8f", "sha256:a856869c7ff079ad84a3e19cd87a64998350c2b94e9e08e44270faef33400f81" ], "version": "==2.6.11" }, "markdown-include": { "hashes": [ "sha256:72a45461b589489a088753893bc95c5fa5909936186485f4ed55caa57d10250f" ], "index": "pypi", "version": "==0.5.1" }, "markupsafe": { "hashes": [ "sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665" ], "version": "==1.0" }, "mdx-truly-sane-lists": { "hashes": [ "sha256:f00be6d525ab3e4e4b45d96368853b0769c51c1512f262bf02c36b5f618dd013" ], "index": "pypi", "version": "==1.0" }, "mkdocs": { "hashes": [ "sha256:39b993183dac9654d1b5eaa093be2d059aae5d89e0badf480b447a646c725d02", "sha256:7709daf71f737e03813b39d7d2333ad4819247c00aed598c018a3ff66fe1f691" ], "index": "pypi", "version": "==0.15.3" }, "mkdocs-bootstrap": { "hashes": [ "sha256:51482d9869af5838b3fe3bb4b0dd7a86bb055031dab970e39649b52ac8ade5b0", "sha256:ba5dbf3928b3dcaf6c5ffb0fdf915c493b0dd405138d63142511402d6f662bc8" ], "version": "==0.2.0" }, "mkdocs-bootswatch": { "hashes": [ "sha256:4efadca1d3919f02515ed1135221b46be0af94f4b3f99e5f001404d371006574", "sha256:f82f8fb5538f2b32691d3bb93dde1430d63c4127331cad3da440a9281e025891" ], "version": "==0.5.0" }, "pymdown-extensions": { "hashes": [ "sha256:1832c74744119a08f17a1f46fe5cba6b47cbe44f118d5daa1df9da5a6ff801a7", "sha256:cceb5ffbfff24fc29b13d11b8e211040a5d296ce2889eb8f75b64518b5edf3bc" ], "index": "pypi", "version": "==1.1" }, "pyyaml": { "hashes": [ "sha256:0c507b7f74b3d2dd4d1322ec8a94794927305ab4cebbe89cc47fe5e81541e6e8", "sha256:16b20e970597e051997d90dc2cddc713a2876c47e3d92d59ee198700c5427736", "sha256:3262c96a1ca437e7e4763e2843746588a965426550f3797a79fca9c6199c431f", "sha256:326420cbb492172dec84b0f65c80942de6cedb5233c413dd824483989c000608", "sha256:4474f8ea030b5127225b8894d626bb66c01cda098d47a2b0d3429b6700af9fd8", "sha256:592766c6303207a20efc445587778322d7f73b161bd994f227adaa341ba212ab", "sha256:5ac82e411044fb129bae5cfbeb3ba626acb2af31a8d17d175004b70862a741a7", "sha256:5f84523c076ad14ff5e6c037fe1c89a7f73a3e04cf0377cb4d017014976433f3", "sha256:827dc04b8fa7d07c44de11fabbc888e627fa8293b695e0f99cb544fdfa1bf0d1", "sha256:b4c423ab23291d3945ac61346feeb9a0dc4184999ede5e7c43e1ffb975130ae6", "sha256:bc6bced57f826ca7cb5125a10b23fd0f2fff3b7c4701d64c439a300ce665fff8", "sha256:c01b880ec30b5a6e6aa67b09a2fe3fb30473008c85cd6a67359a1b15ed6d83a4", "sha256:ca233c64c6e40eaa6c66ef97058cdc80e8d0157a443655baa1b2966e812807ca", "sha256:e863072cdf4c72eebf179342c94e6989c67185842d9997960b3e69290b2fa269" ], "version": "==3.12" }, "six": { "hashes": [ "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9", "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb" ], "version": "==1.11.0" }, "tornado": { "hashes": [ "sha256:1b83d5c10550f2653380b4c77331d6f8850f287c4f67d7ce1e1c639d9222fbc7", "sha256:408d129e9d13d3c55aa73f8084aa97d5f90ed84132e38d6932e63a67d5bec563", "sha256:88ce0282cce70df9045e515f578c78f1ebc35dcabe1d70f800c3583ebda7f5f5", "sha256:ba9fbb249ac5390bff8a1d6aa4b844fd400701069bda7d2e380dfe2217895101", "sha256:c050089173c2e9272244bccfb6a8615fb9e53b79420a5551acfa76094ecc3111" ], "version": "==5.0.2" } }, "develop": {} } ```
januszm commented 3 years ago

This is one of the most confusing parts of pipenv and I think it should be addressed. The install command, instead of taking exactly what's in the .lock file and just installing it, it takes the list from Pipefile and updates the Pipfile.lock. This is just my personal opinion, but I think most people would expect install command to install versions from .lock and update command to update them. This is in contradiction to how similar tools work in other languages.

anyways, pipenv sync is the command that installs versions with lock. pipenv sync --dev additionally installs development dependencies.

uranusjr commented 3 years ago

I agree this is quite confusing if you come from the pip freeze/pip install -r workflow, but…

This is in contradiction to how similar tools work in other languages.

yarn install

  • If yarn.lock is present and is enough to satisfy all the dependencies listed in package.json, the exact versions recorded in yarn.lock are installed, and yarn.lock will be unchanged. Yarn will not check for newer versions.
  • If yarn.lock is absent, or is not enough to satisfy all the dependencies listed in package.json (for example, if you manually add a dependency to package.json), Yarn looks for the newest versions available that satisfy the constraints in package.json. The results are written to yarn.lock.

cargo install

By default, the Cargo.lock file that is included with the package will be ignored. This means that Cargo will recompute which versions of dependencies to use, possibly using newer versions that have been released since the package was published. The --locked flag can be used to force Cargo to use the packaged Cargo.lock file if it is available.

I understand the behaviour may not be straightforward if you’re used to a certain workflow. Full disclosure: I don’t like the current pipenv install behaviour myself. But to say its a constradiction to other languages is, uh… inaccurate, let’s put it that way.

januszm commented 3 years ago

Ok, "contradiction" is a bit too much, but I think you can see where I'm going with this.

I don't think cargo is doing it right too, but the yarn action matches what I wrote and above all it is worth adding the bundle install description, bundler can be used as a model implementation of the dependency resolver to be honest:

bundle install

If a Gemfile.lock does exist, and you have not updated your Gemfile(5), Bundler will fetch all remote sources, but use the dependencies specified in the Gemfile.lock instead of resolving dependencies. If a Gemfile.lock does exist, and you have updated your Gemfile(5), Bundler will use the dependencies in the Gemfile.lock for all gems that you did not update, but will re-resolve the dependencies of gems that you did update.

update is a "change", it means we can add something potentially harmful, but for sure untested, to the project. According to the principle of least surprise, by running install we should get exactly the effect that was approved in the project in the form of a lock file.

revolter commented 3 years ago

Also CocoaPods:

pod install

  • Every time the pod install command is run — and downloads and install new pods — it writes the version it has installed, for each pods, in the Podfile.lock file. This file keeps track of the installed version of each pod and locks those versions.
  • When you run pod install, it only resolves dependencies for pods that are not already listed in the Podfile.lock.
    • For pods listed in the Podfile.lock, it downloads the explicit version listed in the Podfile.lock without trying to check if a newer version is available
    • For pods not listed in the Podfile.lock yet, it searches for the version that matches what is described in the Podfile (like in pod 'MyPod', '~>1.2')
wd60622 commented 3 years ago

Experienced this issue while trying to install package and previously installed packages were upgraded without awareness.

In my scenario, using pipenv install jupyter also upgraded my pymssql from 2.1.x to 2.2.0 which then caused breaking changes with mac os x. This took some debugging but landed on enforcing the version on pymssql~=2.1.4

matteius commented 2 years ago

This is expected that pipenv install will upgrade to the latest packages that your Pipfile specifiers will allow.