search

pypa / pipenv

Python Development Workflow for Humans.
https://pipenv.pypa.io
MIT License
24.81k stars 1.86k forks source link

`--extra-pip-args` doesn't pick up `--platform` option for `pip` correclty. #6001

Closed ChannyClaus closed 3 months ago

ChannyClaus commented 10 months ago

Issue description

--platform flag (along with --no-deps and --target) from pip doesn't seem to get picked up correctly by pipenv via --extra-pip-args.

Expected result

The installation should succeed since pip install does with those flags:

$ pip install --platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/ tensorflow~=2.8.0
Collecting tensorflow~=2.8.0
  Using cached tensorflow-2.8.4-cp38-cp38-macosx_10_14_x86_64.whl (217.9 MB)
Installing collected packages: tensorflow
Successfully installed tensorflow-2.8.4
WARNING: Target directory /Users/chan.kang/.local/share/virtualenvs/tensorflow-4pbUTlUx/lib/python3.8/site-packages/tensorflow-2.8.4.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/chan.kang/.local/share/virtualenvs/tensorflow-4pbUTlUx/lib/python3.8/site-packages/tensorflow already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/chan.kang/.local/share/virtualenvs/tensorflow-4pbUTlUx/lib/python3.8/site-packages/bin already exists. Specify --upgrade to force replacement.

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: python3.8 -m pip install --upgrade pip

Actual result

$ pipenv install --verbose --extra-pip-args "--platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/"  tensorflow~=2.8.0
Installing tensorflow~=2.8.0...
Resolving tensorflow~=2.8.0...
Added tensorflow to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock (db4242) out of date, updating to (904453)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.star
ting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.addi
ng_requirement(SpecifierRequirement('tensorflow~=2.8.0'), None)
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not 
find a version that satisfies the requirement tensorflow~=2.8.0 (from 
versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1)
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pipenv.patched.pip._vendor.resolvelib.resolvers.RequirementsConflicted: 
Requirements conflict: SpecifierRequirement('tensorflow~=2.8.0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_internal/resolutio
n/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/resolvelib/
resolvers.py", line 399, in resolve
    raise ResolutionImpossible(e.criterion.information)
pipenv.patched.pip._vendor.resolvelib.resolvers.ResolutionImpossible: 
[RequirementInformation(requirement=SpecifierRequirement('tensorflow~=2.8.0'),
parent=None)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 440, 
in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/patched/pip/_internal/resolutio
n/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pipenv.patched.pip._internal.exceptions.DistributionNotFound: No matching 
distribution found for tensorflow~=2.8.0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 675, in 
<module>
    main()
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 661, in main
    _main(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 645, in 
_main
    resolve_packages(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 612, in 
resolve_packages
    results, resolver = resolve(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/resolver.py", line 592, in 
resolve
    return resolve_deps(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 908, 
in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 681, 
in actually_resolve_deps
    resolver.resolve()
  File 
"/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/V
ersions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 442, 
in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: [31m[1mERROR[0m: [33mNo matching 
distribution found for tensorflow~=2.8.0[0m
✘ Locking Failed!
⠧ Locking...
Traceback (most recent call last):
  File "/Users/chan.kang/.pyenv/versions/3.8.17/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/cli/command.py", line 209, in install
    do_install(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/install.py", line 297, in do_install
    raise e
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/install.py", line 281, in do_install
    do_init(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/install.py", line 648, in do_init
    do_lock(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/routines/lock.py", line 65, in do_lock
    venv_resolve_deps(
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 849, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 718, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

When possible, provide the verbose output (--verbose), especially for locking and dependencies resolving issues.

Steps to replicate

Running the commands in the above snippet in an empty directory is sufficient to reproduce this issue.

$ pipenv --support Pipenv version: `'2023.10.24'` Pipenv location: `'/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pipenv'` Python location: `'/Users/chan.kang/.pyenv/versions/3.8.17/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8'` OS Name: `'posix'` User pip version: `'23.3.1'` user Python installations found: PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.8.17', 'os_name': 'posix', 'platform_machine': 'arm64', 'platform_python_implementation': 'CPython', 'platform_release': '22.6.0', 'platform_system': 'Darwin', 'platform_version': 'Darwin Kernel Version 22.6.0: Wed Jul 5 22:21:53 PDT ' '2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6020', 'python_full_version': '3.8.17', 'python_version': '3.8', 'sys_platform': 'darwin'} ``` System environment variables: - `SHELL` - `LSCOLORS` - `PYENV_HOOK_PATH` - `GITLAB_PYPI_USERNAME` - `PYENV_SHELL` - `BOKEH_SIGN_SESSIONS` - `XPC_FLAGS` - `NVM_INC` - `TERM_PROGRAM_VERSION` - `REACT_APP_RECAPTCHA_SITE_KEY` - `GITLAB_PYPI_PROJECT_ID` - `__CFBundleIdentifier` - `SSH_AUTH_SOCK` - `TERM_SESSION_ID` - `HOMEBREW_PREFIX` - `SF_WAREHOUSE` - `GITLAB_PYPI_PASSWORD` - `PYENV_VERSION` - `PWD` - `LOGNAME` - `STAGE` - `LLVM_CONFIG` - `SF_USERNAME` - `MANPATH` - `RDS_PASSWORD` - `LaunchInstanceID` - `LDFLAGS` - `HOME` - `RDS_DB_NAME` - `LANG` - `SECURITYSESSIONID` - `UNLEASH_INSTANCE_ID` - `SF_ROLE` - `TMPDIR` - `CPPFLAGS` - `CLICOLOR` - `LOCAL_GRAPHQL_DEV` - `SF_DATABASE` - `PYENV_DIR` - `UNLEASH_URL` - `INFOPATH` - `RDS_PORT` - `NVM_DIR` - `RDS_HOSTNAME` - `TERM` - `SF_ACCOUNT` - `GITLAB_READ_PACKAGE_REGISTRY_TOKEN` - `USER` - `HOMEBREW_CELLAR` - `RDS_USERNAME` - `SHLVL` - `NVM_CD_FLAGS` - `HOMEBREW_REPOSITORY` - `SF_PASSWORD` - `XPC_SERVICE_NAME` - `PYENV_ROOT` - `SF_SCHEMA` - `PATH` - `BOKEH_SECRET_KEY` - `NVM_BIN` - `RECAPTCHA_SECRET_KEY` - `BOKEH_PY_LOG_LEVEL` - `OLDPWD` - `TERM_PROGRAM` - `__CF_USER_TEXT_ENCODING` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/Users/chan.kang/.pyenv/versions/3.8.17/bin:/Users/chan.kang/.pyenv/versions/3.8.17/bin:/opt/homebrew/Cellar/pyenv/2.3.23/libexec:/opt/homebrew/Cellar/pyenv/2.3.23/plugins/python-build/bin:/Users/chan.kang/.nvm/versions/node/v18.16.1/bin:/Users/chan.kang/.pyenv/shims:/opt/homebrew/opt/postgresql@15/bin:/opt/homebrew/opt/llvm/bin:/opt/homebrew/opt/findutils/libexec/gnubin:/opt/homebrew/opt/libpq/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/Users/chan.kang/.cargo/bin:/Users/chan.kang/.gem/ruby/X.X.0/bin:/usr/local/opt/openjdk/bin:/Users/chan.kang/.local/bin:/opt/homebrew/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/local/opt/grep/libexec/gnubin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin` - `SHELL`: `/opt/homebrew/Cellar/bash/5.2.15/bin/bash` - `LANG`: `en_US.UTF-8` - `PWD`: `/Users/chan.kang/test/tensorflow2` --------------------------- Contents of `Pipfile` ('/Users/chan.kang/test/tensorflow2/Pipfile'): ```toml [[source]] url = "https://pypi.org/simple" verify_ssl = true name = "pypi" [packages] [dev-packages] [requires] python_version = "3.8" ```
ChannyClaus commented 10 months ago

took a quick look at the code - seems like this flag isn't passed down to the locking function https://github.com/pypa/pipenv/blob/main/pipenv/routines/install.py#L648-L655

is pipenv not designed to handle the case where the user is trying to masquerade a platform different from the one they are running pipenv on? (i'm trying to create a lock for Pipfile where the dependencies exist only for x86-64, i.e. not for arm64)

matteius commented 10 months ago

@ChannyClaus I opened a PR that I believe addresses it. https://github.com/pypa/pipenv/pull/6006 I don't think you'll need to pass --no-deps since we already pass this as part of pipenv installation.

ChannyClaus commented 10 months ago

@ChannyClaus I opened a PR that I believe addresses it. #6006 I don't think you'll need to pass --no-deps since we already pass this as part of pipenv installation.

hmm does the command above pipenv install --verbose --extra-pip-args "--platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/" tensorflow~=2.8.0 work with the change? i just tried it out locally and it seems to be failing still.

matteius commented 10 months ago

Well I wouldn't pass --no-deps because as I said we already pass that by default, but I wasn't able to get it to work with the specifier ~=2.8.0 but removing that it did work.

ChannyClaus commented 10 months ago

yeah i think making it work with the specifier is critical - at least for the use case we have a bunch of apple silicon macbook, i.e.

In [1]: import platform

In [2]: platform.machine()
Out[2]: 'arm64'

users working with a Pipfile that pins tensorflow at ~=2.8.0. not exactly sure how pipenv lock works under the hood but if it's using pip under the hood it supposedly should work since pip install tensorflow~=2.8.0 works with the flags in the PR description.

matteius commented 10 months ago

Well something is wrong with that version for it to be resolved, version specifiers in generally are working.

matteius commented 10 months ago

Actually my reason it wasn't resolving is I have default python 3.11 and that version only goes up to 3.11 -- when I run this it works:

$ pipenv install --verbose --extra-pip-args "--platform=win_amd64 --target C:\\c\\Users\\matte\\.virtualenvs\\issue-6001-WKuyds7q\\Lib\\site-packages\\"  tensorflow~=2.8.0 --python 3.10
matteius commented 10 months ago

The other thing my PR does not address is how to store extra pip args in the Pipfile for future relocking so it doesn't have to be supplied on the CLI everytime. Also I am pretty sure in my PR i haven't exposed it directly to the lock command yet but that should be easy enough to add.

ChannyClaus commented 10 months ago 
$ git fetch
g(pipenv) chan.kang@Chans-MBP-743 ~/repos/pipenv -  (issue-6001-extra-pip-args)
$ git reset --hard origin/issue-6001-extra-pip-args
HEAD is now at 12278956 Use the extra pip args in the resolver.
(pipenv) chan.kang@Chans-MBP-743 ~/repos/pipenv -  (issue-6001-extra-pip-args)
$  pipenv install --verbose --extra-pip-args "--platform=win_amd64 --target test"  tensorflow~=2.8.0 --python 3.10
Installing tensorflow~=2.8.0...
Resolving tensorflow~=2.8.0...
Added tensorflow to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock (f2a69e) out of date, updating to (5bfae3)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('pytz'), None)
INFO:pipenv.patched.pip._internal.operations.prepare:Collecting pytz (from -r 
/var/folders/s4/dtm89b1x7zxcy0ly0frk041c0000gq/T/pipenv-e2w65hf3-requirements/pipenv-ydqbue5q-constraints.txt (line 4))
INFO:pipenv.patched.pip._internal.network.download:Using cached pytz-2023.3.post1-py2.py3-none-any.whl.metadata (22 kB)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('tensorflow~=2.8.0'
), None)
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement 
tensorflow~=2.8.0 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1)
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pipenv.patched.pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: 
SpecifierRequirement('tensorflow~=2.8.0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 399, in resolve
    raise ResolutionImpossible(e.criterion.information)
pipenv.patched.pip._vendor.resolvelib.resolvers.ResolutionImpossible: 
[RequirementInformation(requirement=SpecifierRequirement('tensorflow~=2.8.0'), parent=None)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 444, in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pipenv.patched.pip._internal.exceptions.DistributionNotFound: No matching distribution found for tensorflow~=2.8.0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 675, in <module>
    main()
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 661, in main
    _main(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 645, in _main
    resolve_packages(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 612, in resolve_packages
    results, resolver = resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 592, in resolve
    return resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 917, in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 685, in actually_resolve_deps
    resolver.resolve()
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 446, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: [31m[1mERROR[0m: [33mNo matching distribution found for tensorflow~=2.8.0[0m
✘ Locking Failed!
⠧ Locking packages...
Traceback (most recent call last):
  File "/Users/chan.kang/.local/share/virtualenvs/pipenv-Ex9cqLYG/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/command.py", line 209, in install
    do_install(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 297, in do_install
    raise e
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 281, in do_install
    do_init(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 648, in do_init
    do_lock(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/lock.py", line 66, in do_lock
    venv_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 858, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 722, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

hmm something still seems awry on my end (just tried the command that worked for you); maybe i'm doing something wrong? (it seems to be using the correct code based on the stacktrace)

matteius commented 10 months ago

What other packages in your Pipfile dependencies might be affecting your resolution? maybe something is requiring a newer version of tensorflow.

matteius commented 10 months ago

Well wait, win_amd64 isn't going to work for you on a MBP ...

matteius commented 10 months ago

Looks like the wheel you are targeting is [macosx_10_14_x86_64] as you had in your original comment (https://files.pythonhosted.org/packages/b1/c7/0df8a0819e16af0fdd38791dc048af621a089fb952e313206bb9e6fea42d/tensorflow-2.8.3-cp310-cp310-macosx_10_14_x86_64.whl)

ChannyClaus commented 10 months ago

Well wait, win_amd64 isn't going to work for you on a MBP ...

well that was actually kinda why i created this issue - to be able to feed platforms different from my local when installing / locking (our use case is for masquarading x86-64 but the implementation should work for all platforms, i suspect?). for example, pip install works:

$ pip install tensorflow~=2.8.0 --platform=win_amd64 --target test --no-deps
Collecting tensorflow~=2.8.0
  Downloading tensorflow-2.8.4-cp310-cp310-win_amd64.whl (438.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 438.4/438.4 MB 9.6 MB/s eta 0:00:00
Installing collected packages: tensorflow
Successfully installed tensorflow-2.8.4

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
chan.kang@Chans-MBP-743 ~/repos/pipenv -  (issue-6001-extra-pip-args)
$

and i was hoping we'd be able to get pipenv to work with it as well.

matteius commented 10 months ago

Well you haven't answered my question -- do you have anything else in your Pipfile being specified? I showed an example of it working with my branch, but I had to downgrade from python 3.11 -- however your logs indicate something else in the Pipfile is restricting the resolver to a version of tensorflow higher than what you are installing. Installing it as a 1-off with pip isn't the same check if you have anything else defined in your Pipfile.

ChannyClaus commented 10 months ago

sorry about that. fails the same way in a newly created dir:

$ rm -rf new_dir && mkdir new_dir && cd new_dir && pipenv install --verbose --extra-pip-args "--platform=win_amd64 --target test"  tensorflow~=2.8.0 --python 3.10
Creating a Pipfile for this project...
Installing tensorflow~=2.8.0...
Resolving tensorflow~=2.8.0...
Added tensorflow to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('tensorflow~=2.8.0
'), None)
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement 
tensorflow~=2.8.0 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1, 2.14.0rc0, 2.14.0rc1, 2.14.0, 2.15.0rc0, 
2.15.0rc1)
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pipenv.patched.pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: 
SpecifierRequirement('tensorflow~=2.8.0')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 399, in resolve
    raise ResolutionImpossible(e.criterion.information)
pipenv.patched.pip._vendor.resolvelib.resolvers.ResolutionImpossible: 
[RequirementInformation(requirement=SpecifierRequirement('tensorflow~=2.8.0'), parent=None)]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 444, in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 104, in resolve
    raise error from e
pipenv.patched.pip._internal.exceptions.DistributionNotFound: No matching distribution found for tensorflow~=2.8.0
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 675, in <module>
    main()
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 661, in main
    _main(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 645, in _main
    resolve_packages(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 612, in resolve_packages
    results, resolver = resolve(
  File "/Users/chan.kang/repos/pipenv/pipenv/resolver.py", line 592, in resolve
    return resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 917, in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 685, in actually_resolve_deps
    resolver.resolve()
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 446, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: [31m[1mERROR[0m: [33mNo matching distribution found for tensorflow~=2.8.0[0m
✘ Locking Failed!
⠸ Locking packages...
Traceback (most recent call last):
  File "/Users/chan.kang/.local/share/virtualenvs/pipenv-Ex9cqLYG/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/chan.kang/repos/pipenv/pipenv/cli/command.py", line 209, in install
    do_install(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 297, in do_install
    raise e
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 281, in do_install
    do_init(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/install.py", line 672, in do_init
    do_lock(
  File "/Users/chan.kang/repos/pipenv/pipenv/routines/lock.py", line 66, in do_lock
    venv_resolve_deps(
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 858, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/Users/chan.kang/repos/pipenv/pipenv/utils/resolver.py", line 722, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!
(pipenv) chan.kang@Chans-MBP-755 ~/new_dir - 
$
matteius commented 10 months ago

@ChannyClaus you are still specifying a windows platform in that example. Digging a bit deeper, you have a arm64 architecture, but the 2.8 wheels do not have a mac os arm wheel (just x86 as tensorflow-2.8.0-cp310-cp310-macosx_10_14_x86_64.whl or tensorflow-2.8.4-cp310-cp310-macosx_10_14_x86_64.whl is the latest in the 1.8.x chain which is what you are specifying). Even in your example pip is installing the x86 whl, which doesn't seem right for your system (but is what you are specifying). Since there are only prebuilt wheels for this package, no sdist will be built and the pipenv resolver fails.

I am not quite sure why it wouldn't fail with the pip example, but we see that in the stack trace that there are newer wheels that match your system, specifically: CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement tensorflow~=2.8.0 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1, 2.14.0rc0, 2.14.0rc1, 2.14.0, 2.15.0rc0, 2.15.0rc1) which would be the macosx_12_0_arm64.whl.

FWIW, you should be able to simplify the command you are running: pip install --platform=macosx_10_14_x86_64 --no-deps --target $(pipenv --venv)/lib/python3.8/site-packages/ tensorflow~=2.8.0 to pipenv run pip install --platform=macosx_10_14_x86_64 --no-deps tensorflow~=2.8.0 but it sill wouldn't be in the lockfile or Pipfile.