Closed kureta closed 5 years ago
kureta
commented
5 years ago I guess the page isn't a valid index, so the right option to use is --find-links which is apparently only possible via an environment variable. It works with PIP_FIND_LINKS but shouldn't there also be a way to specify this inside the Pipfile? Should I close this issue and open another one for a feature request, or will this be the only official way to do this?
kureta
commented
5 years ago To clarify: I have set the environment variable PIP_FIND_LINKS="https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html" and now all the pipenv commands work without a problem. It is able to find and install the latest torch-nightly. However this environment variable cannot be seen in Pipfile or Pipfile.lock and there is no way for anyone else who clones my repository to know that they have to set this environment variable, other than adding it explicitly to setup instructions, which is less than ideal. And my latest configuration is as follows:
frostming
commented
5 years ago @kureta If environment variables can work why would we add a section to Pipfile? In fact, the core developers are against making Pipfile a config file for Pipenv tool. It should just contain what is needed for reproducing an environment.
[source] section works like --index-url and --extra-index-url
I will close it for now. Thanks.
kureta
commented
5 years ago @frostming I tired using [source] section and it couldn't find the module. This is not about configuration, it is exactly about reproducing an environment. It should work under the [source] section but it doesn't. Please read my first comment again.
kureta
commented
5 years ago It is impossible to reproduce this environment without knowing that torch-nightly can only be obtained from "https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html", and adding
[[source]]
name = "pytorch-nightly"
url = "https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html"
verify_ssl = trueto Pipfile should work but doesn't. I am not trying to be a pain but I really fail to see how this is a configuration issue.
frostming
commented
5 years ago @kureta I emphasized in my last comment that [source] section works like --index-url not --find-links, so it is normal that adding a non-index link to source won't work.
To make the Pipfile itself reproducible, you can specify the dependency with file url like this:
[packages]
torch-nightly = {file = "https://download.pytorch.org/whee/download/link.whl"}Is this possible?
kureta
commented
5 years ago @frostming thank you very much. Your solution works perfectly!
ajrcampbell
commented
5 years ago @kureta how did you manage to install the nightly build of PyTorch with pipenv?
kureta
commented
5 years ago @ajrcampbell just go to https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html and view source code. Download the latest .whl from the list and add it to your Pipfile like this
[packages]
torch-nightly = {file = "https://download.pytorch.org/whee/download/link.whl"}Or add this environment variable:
PIP_FIND_LINKS="https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html"and add torch-nightly to your Pipfile
[packages]
torch-nightly = "*"However, torch is a very large package and, for me, it took a lot of time to lock and/or install anything while I had torch-nightly in my Pipfile, so I downloaded the latest wheel and added it as a local file to like so:
[packages]
torch-nightly = {file = "~/Downloads/torch.whl"}It still took a long time to lock and I finally moved on to venv and pip.
Issue description
Pytorch nightly can be installed using this
pipcommand:I am unable to install the same package using
pipenv install.I added this
sourceentry in myPipfileand added
torch-nightlyunderpackageslike this:Expected result
Installation of
torch-nightlyfrom pytorch's indexActual result
Steps to replicate
Add the above
[[source]]and[packages]entries in yourPipfileand runpipenv install$ pipenv --support
Pipenv version: `'2018.11.15.dev0'` Pipenv location: `'/usr/lib/python3.7/site-packages/pipenv'` Python location: `'/usr/bin/python'` Python installations found: - `3.7.3`: `/usr/bin/python3` - `3.7.3`: `/usr/bin/python3.7m` - `3.7.2`: `/home/kureta/.pyenv/versions/dimensions-venv/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/exif-move-venv/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/DRIT-venv/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/tf-venv/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/wgan-gp-venv/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/zachary-venv/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/stylegan-venv/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/3.7.2/bin/python` - `3.7.2`: `/home/kureta/.pyenv/versions/3.7.2/bin/python3.7m` - `2.7.16`: `/usr/bin/python2` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.7.3', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.19.31-rt18-4-rt-bfq', 'platform_system': 'Linux', 'platform_version': '#1 SMP PREEMPT RT Sat Apr 13 17:52:33 +03 2019', 'python_full_version': '3.7.3', 'python_version': '3.7', 'sys_platform': 'linux'} ``` System environment variables: - `SHELL` - `DESKTOP_SESSION` - `GTK_MODULES` - `XDG_SEAT` - `PWD` - `LOGNAME` - `QT_QPA_PLATFORMTHEME` - `XDG_SESSION_TYPE` - `DSSI_PATH` - `XAUTHORITY` - `HOME` - `LANG` - `VST_PATH` - `XDG_CURRENT_DESKTOP` - `INVOCATION_ID` - `XDG_SESSION_CLASS` - `USER` - `DISPLAY` - `SHLVL` - `XDG_VTNR` - `XDG_SESSION_ID` - `XDG_RUNTIME_DIR` - `MAVEN_OPTS` - `JOURNAL_STREAM` - `PATH` - `DBUS_SESSION_BUS_ADDRESS` - `LV2_PATH` - `MAIL` - `LADSPA_PATH` - `CADENCE_AUTO_STARTED` - `_` - `KITTY_WINDOW_ID` - `WINDOWID` - `TERM` - `COLORTERM` - `TERMINFO` - `OLDPWD` - `ZSH` - `GEM_HOME` - `EDITOR` - `ALTERNATE_EDITOR` - `PAGER` - `LESS` - `LSCOLORS` - `LS_COLORS` - `PYENV_SHELL` - `LC_CTYPE` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/kureta/.pyenv/shims:/home/kureta/.gem/ruby/2.6.0/bin:/home/kureta/.local/bin:/home/kureta/.gem/ruby/2.6.0/bin:/home/kureta/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/cuda/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/kureta/.pyenv/bin` - `SHELL`: `/usr/bin/zsh` - `EDITOR`: `vim` - `LANG`: `en_US.UTF-8` - `PWD`: `/home/kureta/Documents/repos/berlin` --------------------------- Contents of `Pipfile` ('/home/kureta/Documents/repos/berlin/Pipfile'): ```toml [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [[source]] name = "pytorch" url = "https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html" verify_ssl = true [dev-packages] [packages] numpy = "*" opencv-python = "*" essentia = "*" jack-client = "*" torch-nightly = {version="*", index="pytorch"} torchvision-nightly = "*" [requires] python_version = "3.7" [pipenv] allow_prereleases = true ``` Contents of `Pipfile.lock` ('/home/kureta/Documents/repos/berlin/Pipfile.lock'): ```json { "_meta": { "hash": { "sha256": "463d71343037600bdf8b22cda39a97f0e9cc6cd9cd9bf65dc299658435997b81" }, "pipfile-spec": 6, "requires": { "python_version": "3.7" }, "sources": [ { "name": "pypi", "url": "https://pypi.org/simple", "verify_ssl": true }, { "name": "pytorch", "url": "https://download.pytorch.org/whl/nightly/cu100/torch_nightly.html", "verify_ssl": true } ] }, "default": {}, "develop": {} } ```