`pipenv install XYZ` command sometimes hangs indefinitely at "Locking [packages] dependencies"

[MarenstainBear]

Similar to these issues:

• https://github.com/pypa/pipenv/issues/356
• https://github.com/pypa/pipenv/issues/278

Summary

When using pipenv install XYZ, the process hangs indefinitely at "Locking [packages] dependencies".

This doesn't happen every time, and may be related to which package is being installed. This morning it is happening every time I try to install tensorflow. Last night it was happening for numpy as well, but today numpy is fine.

The hanged response seems to last indefinitely (though I did manually terminate the process after an hour). The Pipfile.lock is not written, but the Pipfile is written and the expected packages are installed and listed when I run pipenv run pip freeze.

I was able to reproduce this using various permutations of:

• (L)ubuntu 16.04 and (X)ubuntu 17.10
• pipenv --three and pipenv --two
• pipenv installed through python 2.7 and pipenv installed through python 3.5
• ...and I downloaded the pipenv source and installed that in an isolated environment to see if the issue was specific to pipenv version 11.8.3. I experienced the same behavior in 11.9.0.

I am also able to reproduce it by simply running pipenv lock or pipenv update if I have tensorflow (or whatever package is currently causing me trouble) listed in my Pipfile.

$ python -m pipenv.help output

Pipenv version: `'11.8.3'` Pipenv location: `'/home/mary/.local/lib/python2.7/site-packages/pipenv'` Python location: `'/usr/bin/python'` Other Python installations in `PATH`: - `2.7`: `/usr/bin/python2.7` - `2.7`: `/usr/bin/python2.7` - `3.5`: `/usr/bin/python3.5m` - `3.5`: `/usr/bin/python3.5` - `2.7.12`: `/usr/bin/python` - `2.7.12`: `/usr/bin/python2` - `3.5.2`: `/usr/bin/python3` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '0', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.13.0-37-generic', 'platform_system': 'Linux', 'platform_version': '#42~16.04.1-Ubuntu SMP Wed Mar 7 16:03:28 UTC 2018', 'python_full_version': '2.7.12', 'python_version': '2.7', 'sys_platform': 'linux2'} ``` System environment variables: - `MANDATORY_PATH` - `_LXSESSION_PID` - `XDG_GREETER_DATA_DIR` - `PROJECT_HOME` - `LC_CTYPE` - `PYTHONDONTWRITEBYTECODE` - `XDG_CURRENT_DESKTOP` - `XDG_SESSION_TYPE` - `LOGNAME` - `XDG_SEAT` - `PATH` - `XDG_VTNR` - `QT_PLATFORM_PLUGIN` - `PYTHONUNBUFFERED` - `VIRTUALENVWRAPPER_SCRIPT` - `ZSH` - `DISPLAY` - `SSH_AGENT_PID` - `LANG` - `TERM` - `SHELL` - `XDG_SESSION_PATH` - `XAUTHORITY` - `LANGUAGE` - `SHLVL` - `QT_LINUX_ACCESSIBILITY_ALWAYS_ON` - `QT_QPA_PLATFORMTHEME` - `SESSION_FOLDER` - `QT_ACCESSIBILITY` - `WINDOWID` - `LIBVIRT_DEFAULT_URI` - `HOME` - `XDG_SESSION_DESKTOP` - `SAL_USE_VCLPLUGIN` - `XDG_RUNTIME_DIR` - `WORKON_HOME` - `SSH_AUTH_SOCK` - `VTE_VERSION` - `GDMSESSION` - `VIRTUALENVWRAPPER_WORKON_CD` - `XDG_SEAT_PATH` - `PIP_PYTHON_PATH` - `XDG_SESSION_ID` - `DBUS_SESSION_BUS_ADDRESS` - `_` - `VIRTUALENVWRAPPER_HOOK_DIR` - `VIRTUALENVWRAPPER_PROJECT_FILENAME` - `DESKTOP_SESSION` - `LSCOLORS` - `XDG_CONFIG_DIRS` - `DEFAULTS_PATH` - `XDG_CONFIG_HOME` - `OLDPWD` - `LS_COLORS` - `GDM_LANG` - `XDG_DATA_DIRS` - `PWD` - `XDG_MENU_PREFIX` - `LESS` - `PAGER` - `USER` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/mary/bin:/home/mary/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games` - `SHELL`: `/usr/bin/zsh` - `LANG`: `en_US.UTF-8` - `PWD`: `/home/mary/Development/Projects/poor_mans_smart_camera` --------------------------- Contents of `Pipfile` ('/home/mary/Development/Projects/poor_mans_smart_camera/Pipfile'): ```toml [[source]] url = "https://pypi.python.org/simple" verify_ssl = true name = "pypi" [dev-packages] [packages] numpy = "*" tensorflow = "*" [requires] python_version = "3.5" ```

---

Expected result

I expect the step in which the Pipfile.lock is generated to take at most a minute.

I have good hardware with lots of free memory and processing power, so I don't think this is a resource constraint. My internet speed is 60 Mbps, so I doubt it is a networking issue when requesting packages.

Actual result

verbose_output.txt

While this screenshot doesn't show verbose output like the file above, it does visually demonstrate what I am experiencing.

Steps to replicate

1. Spin up new virtualenv using pipenv --three or pipenv --two.
2. Confirm things are working as expected with a package you know is installing fine such as pipenv install pylint.
3. Now cause it to hang by installing a package that is not working for whatever reason such as pipenv install tensorflow.

Pipfile

[uranusjr]

Not that this should happen in any case, but what hardware are you running this on?

[MarenstainBear]

@uranusjr Please find the attached hardware information report. hardinfo_report.txt

[techalchemy]

Does this happen consistently? Try pipenv lock —verbose —clear

[MarenstainBear]

@techalchemy, I ran pipenv lock --verbose --clear, and the results are similar:

> pipenv lock --verbose --clear
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1                           
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done

Locking [packages] dependencies…

It remained hung at "Locking [packages] dependencies".

Any luck reproducing this on your machine? With tensorflow in particular, it is happening every time for me.

---

EDIT: I tested this out on a friend's computer. He's running Ubuntu 17.10, and he did not experience the problem. Both computers are on the same network. One difference between his computer and mine is that he only installed pipenv rather than pipenv and virtualenvwrapper. I'll spin up a virtual machine to see if this is somehow connected to virtualenvwrapper and/or the environment variables I have set that are shared across both (i.e. $WORKON_HOME).

[roddds]

I'm currently seeing the same problem; my pipenv lock --verbose --clear output looks exactly like @MarenstainBear's. I'm running Ubuntu and am using pyenv to manage virtualenvs.

hardinfo_report.txt

[uranusjr]

I see you installed Pipenv against the system Python. Do you happen to have pyopenssl (or something else SSL-related, honestly I can’t remember, it’s part of requests[security]) installed to the same environment? This is known to make the operations extremely slow.

[roddds]

I've installed requests[security] and pyopenssl inside the virtualenv just to make sure, but It's still hanging. Debugging further, I've found that it hangs inside venv_resolve_deps the second time around, specifically in line 376, c = delegator.run(cmd, block=True).

What's weirder is that if I run that command directly (python resolver.py), from the command line, it runs perfectly.

[MarenstainBear]

Yes, it appears as though I do, but only for my python 2, not for python 3 (and my pipenv instance was installed through the python 2 version of pip):

mary@marvel:~$ /usr/bin/python2.7 -m pip freeze | grep -i ssl
pyOpenSSL==17.5.0
mary@marvel:~$ /usr/bin/python3 -m pip freeze | grep -i ssl
mary@marvel:~$ 

I'll go pop over to my other laptop and into some virtual environments where I can't reproduce this to see if they do not have pyOpenSSL.

EDIT: Deleted a couple of paragraphs about another issue I was seeing. That was my error--I was actually in a subdirectory but didn't realize it because without my normal zsh configuration disabled for testing purposes, I no longer had a visual indicator in my prompt of my cwd. As @uranusjr correctly stated "but Pipenv does a lot of environment detection, and is known to work weirdly in an improperly-configured shell."... I indeed had momentarily incorrectly configured my shell.

---

@uranusjr
EDIT 2: I tested this out on a virtual machine with a pretty similar setup to my host machine (including installing pipenv against a python 2.7.12 environment containing the pyOpenSSL module), and on that machine everything is working perfectly as expected. That gives me doubt that pyOpenSSL is the problem here. (I did note that the pipenv version is higher on my virtual machine, so I think I'll upgrade pipenv locally to see if that makes a difference.)

$ python -m pipenv.help output (from the virtual machine)

Pipenv version: `'11.9.0'` Pipenv location: `'/home/mary/.local/lib/python2.7/site-packages/pipenv'` Python location: `'/usr/bin/python'` Other Python installations in `PATH`: - `2.7`: `/usr/bin/python2.7` - `2.7`: `/usr/bin/python2.7` - `3.5`: `/usr/bin/python3.5m` - `3.5`: `/usr/bin/python3.5` - `2.7.12`: `/usr/bin/python` - `2.7.12`: `/usr/bin/python2` - `3.5.2`: `/usr/bin/python3` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '0', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.13.0-36-generic', 'platform_system': 'Linux', 'platform_version': '#40~16.04.1-Ubuntu SMP Fri Feb 16 23:25:58 UTC 2018', 'python_full_version': '2.7.12', 'python_version': '2.7', 'sys_platform': 'linux2'} ``` System environment variables: - `XDG_GREETER_DATA_DIR` - `GNOME_DESKTOP_SESSION_ID` - `PYTHONDONTWRITEBYTECODE` - `LESSOPEN` - `XDG_SESSION_TYPE` - `QT_IM_MODULE` - `LOGNAME` - `USER` - `PROMPT_COMMAND` - `HOME` - `XDG_VTNR` - `PATH` - `PYTHONUNBUFFERED` - `DISPLAY` - `SSH_AGENT_PID` - `LANG` - `TERM` - `SHELL` - `XDG_SESSION_PATH` - `QT_STYLE_OVERRIDE` - `LANGUAGE` - `SESSION_MANAGER` - `SHLVL` - `QT_LINUX_ACCESSIBILITY_ALWAYS_ON` - `GTK_CSD` - `QT_ACCESSIBILITY` - `XMODIFIERS` - `GIO_LAUNCHED_DESKTOP_FILE_PID` - `XDG_SESSION_DESKTOP` - `GIO_LAUNCHED_DESKTOP_FILE` - `XDG_RUNTIME_DIR` - `SSH_AUTH_SOCK` - `VTE_VERSION` - `GDMSESSION` - `XDG_SEAT_PATH` - `LESSCLOSE` - `GSETTINGS_SCHEMA_DIR` - `XDG_CURRENT_DESKTOP` - `XDG_SESSION_ID` - `DBUS_SESSION_BUS_ADDRESS` - `_` - `XAUTHORITY` - `DESKTOP_SESSION` - `XDG_CONFIG_DIRS` - `GTK_MODULES` - `GDM_LANG` - `PANTHEON_TERMINAL_ID` - `XDG_DATA_DIRS` - `PWD` - `PIP_PYTHON_PATH` - `XDG_MENU_PREFIX` - `LS_COLORS` - `XDG_SEAT` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/mary/bin:/home/mary/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games` - `SHELL`: `/bin/bash` - `LANG`: `en_US.UTF-8` - `PWD`: `/home/mary` ---------------------------

[uranusjr]

@MarenstainBear Which Python do you install Pipenv in? Or do you install Pipenv in another Python, or inside a virtualenv? (Not your project, but Pipenv itself.)

The behaviour you mentioned below is indeed curious, but Pipenv does a lot of environment detection, and is known to work weirdly in an improperly-configured shell. I wouldn’t read it too deeply; it’s probably choking on something else.

[uranusjr]

@roddds resolver.py takes the depedencies it wants to resolve as an environment variable. Without setting $PIPENV_PACKAGES before running it it would just resolve an empty list—lightning fast as it should be because there’s nothing to resolve.

Try setting PIPENV_PACKAGES first (the format is similar to requirements.txt) and see what happens.

[jlhood]

+1 I'm seeing the same issue trying to install numpy on MacOS. Verified I don't see this issue installing boto3.

% uname -a
Darwin f45c898a1d21.ant.amazon.com 15.6.0 Darwin Kernel Version 15.6.0: Tue Jan  9 20:12:05 PST 2018; root:xnu-3248.73.5~1/RELEASE_X86_64 x86_64

[roddds]

@uranusjr this is what I get when checking PIPENV_PACKAGES with the same value that was set inside venv_resolve_deps:

Resolver output

$ PIPENV_PACKAGES=='django -i https://pypi.python.org/simple\ngraphene -i https://pypi.python.org/simple\ngraphene-django -i https://pypi.python.org/simple\npsycopg2 -i https://pypi.python.org/simple\npsycopg2-binary -i https://pypi.python.org/simple\nipython -i https://pypi.python.org/simple\ndjango-manifold==0.1.6 -i https://pypi.python.org/simple\nscipy -i https://pypi.python.org/simple\npython-dateutil -i https://pypi.python.org/simple\ndjango-extensions -i https://pypi.python.org/simple' python /home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/resolver.py

Traceback (most recent call last):
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/packaging/requirements.py", line 92, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 1617, in parseString
    raise exc
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 1607, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 3376, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 3698, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 3359, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/pyparsing.py", line 2670, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pip9._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/req/req_install.py", line 82, in __init__
    req = Requirement(req)
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/packaging/requirements.py", line 96, in __init__
    requirement_string[e.loc:e.loc + 8]))
pip9._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'=django'"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/resolver.py", line 83, in 
    main()
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/resolver.py", line 71, in main
    clear=do_clear,
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/resolver.py", line 63, in resolve
    verbose=verbose,
  File "/home/rodrigo/.pyenv/versions/val/lib/python3.6/site-packages/pipenv/utils.py", line 426, in resolve_deps
    pre,
  File "/home/rodrigo/.pyenv/versions/val/lib/python3.6/site-packages/pipenv/utils.py", line 294, in actually_resolve_reps
    c for c in req.parse_requirements(t, session=pip_requests)
  File "/home/rodrigo/.pyenv/versions/val/lib/python3.6/site-packages/pipenv/utils.py", line 294, in 
    c for c in req.parse_requirements(t, session=pip_requests)
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/req/req_file.py", line 93, in parse_requirements
    for req in req_iter:
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/req/req_file.py", line 158, in process_line
    isolated=isolated, options=req_options, wheel_cache=wheel_cache
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/req/req_install.py", line 235, in from_line
    wheel_cache=wheel_cache, constraint=constraint)
  File "/home/rodrigo/.pyenv/versions/3.6.4/envs/val/lib/python3.6/site-packages/pipenv/vendor/pip9/req/req_install.py", line 91, in __init__
    "Invalid requirement: '%s'\n%s" % (req, add_msg))
pip9.exceptions.InstallationError: Invalid requirement: '=django'
= is not a valid operator. Did you mean == ?

My Pipfile

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
django = "*"
graphene = "*"
graphene-django = "*"
"psycopg2" = "*"
"psycopg2-binary" = "*"
ipython = "*"
django-manifold = "==0.1.6"
scipy = "*"
python-dateutil = "*"
django-extensions = "*"

[dev-packages]

[requires]
python_version = "3.6"

[uranusjr]

@roddds You have a superfluous = in the environ. It’s

$ PIPENV_PACKAGES='django -i [too long, snipped]

But you had

$ PIPENV_PACKAGES=='django ...

The second = got combined with django after it, resulting the syntax error you saw.

[slhck]

I thought I was seeing the same problem on a fresh install of a Mac (macOS 10.12, Homebrew Python 3), and found several issues related here.

It's a beefy machine, I have good Internet access – not sure what causes it to take so long, but in the end the lockfile was written after about three minutes.

[Victor-Savu]

Same here. Every time I run pipenv install xxx it takes a very long time (2-3 minutes) right after it prints Locking [packages] dependencies…

[techalchemy]

Locking simply takes a long time folks. Because of python dependency management we have to download each package and run setup.py to get dependencies as needed. For libraries like tensorflow and numpy you may wind up with an sdist which means you are building from source. That is slow. I’ve seen it take 10 minutes. It goes faster if you heat up the cache.

I’m going to close this issue for now until someone can actually show me that their process is hung or not doing anything. We are considering ways to improve this process

[sersorrel]

Would it be feasible to display the package that's currently being locked? I think that would at least help to stop people thinking that Pipenv is hanging.

[slhck]

We are considering ways to improve this process

I'd suggest printing a message that informs the user that $something is being done, which might “take a few minutes”. Right now it looks like the process hangs, which is I guess what gets people to file bug reports or +1 on these (non-)issues.

[Victor-Savu]

@slhck I would be interested to know why this has to take so long to begin with. What kind of work does pipenv need to do in that period?

[techalchemy]

@anowlcalledjosh I like this suggestion, and have probably wondered what was being locked myself. Not sure why we never thought to print it.

@Victor-Savu as I mentioned. compiling something like numpy or tensorflow.

[MarenstainBear]

@techalchemy My process, as far as I can tell, is actually hanging. I've let it go for five hours before killing it. Looking at top, I didn't see any indication that anything was actually being processed. By contrast, when I install tensorflow in my virtualenv with pip instead of pipenv, it takes just a few seconds.

As you can see in this capture of my terminal, it took my system 12.25 seconds to spin up a new python3 virtualenv and use pip to install tensorflow using the command /usr/bin/time -v pipenv run pip install tensorflow.

If I instead run the command /usr/bin/time -v pipenv install tensorflow, the process appears to be hanging indefinitely at the "Locking [packages] dependencies..." after the installation is already successful.

You can see from this image that after about 59 seconds of actively doing something, the subprocess related to resolver.py stops actively using system resources and, as far as I can tell, will never complete. When I took this screenshot, the process had been running for 16 minutes.

[uranusjr]

@MarenstainBear Pipenv spawns a subprocess inside the virtualenv to perform locking. Would you be able to verify if the subprocess (the virtualenv python running pipenv/resolver.py is running when Pipenv stuck? I’m wondering if it is stuck inside the resolver, after the resolver returns, or during delegator.run…

(even more ideally you can try finding out if the resolver reaches its end)

[MarenstainBear]

@uranusjr I inserted some write statements into the resolver.py code, and looks to be hanging somewhere in the pipenv.utils.resolve_deps call at this line.

In other words, resolver.py does not appear to reach its end.

[uranusjr]

@MarenstainBear That’s good(?) to hear, at least it means it’s debuggable. How far did it manage to get inside resolve_deps? It’s a pretty long function…

[techalchemy]

Yeah and for how long was it running? To me that still sounds like what I said before — resolving the dependency graph which is slow

[MarenstainBear]

@techalchemy @uranusjr I'm going to dig into what's going on in the resolve_deps function this weekend. In the meanwhile, here are the results of the process I started yesterday. After 24 hours, it had not resolved, and it really wasn't using any system resources, so I think it was not trying to resolve dependencies and instead just hanging. (I killed the process in order to see the output of the time command.)

Command exited with non-zero status 1 Command being timed: "pipenv install tensorflow" User time (seconds): 13.63 System time (seconds): 1.99 Percent of CPU this job got: 0% Elapsed (wall clock) time (h:mm:ss or m:ss): 25:45:02 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 297772 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 467309 Voluntary context switches: 15781 Involuntary context switches: 207 Swaps: 0 File system inputs: 0 File system outputs: 1654976 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 1

[jerryasher]

@techalchemy what does it mean to "heat up the cache"?

[techalchemy]

@MarenstainBear thanks for doing all that. Your issue is definitely a bug. Others in the thread I am not yet convinced. Can you install from master and see if we have this fixed?

[MarenstainBear]

@techalchemy I installed from master at commit 8a67a21d61a2383253fe0dd5e7a8d79d51d30d2d (dated Sat Mar 31 01:13:26 2018 -0400). My issue was not resolved. I saw the same behavior here in version 11.9.1 as I was seeing in 11.9.0.

I did briefly try a different way to debug this problem using strace. The command strace pipenv install tensorflow hung at poll([{fd=5, events=POLLIN}, {fd=7, events=POLLIN}], 2, -1). When I looked up those file descriptors, there wasn't much of interest going on other than confirmation that the system is waiting for I/O.

mary@marvel:/proc/8471/fd
> lsof -n -P | grep 1428937     
pipenv     8474                  mary    5r     FIFO               0,12       0t0    1428937 pipe
sh         8543                  mary    1w     FIFO               0,12       0t0    1428937 pipe
python     8544                  mary    1w     FIFO               0,12       0t0    1428937 pipe

mary@marvel:/proc/8471/fd
> lsof -n -P | grep 1428938
pipenv     8474                  mary    7r     FIFO               0,12       0t0    1428938 pipe
sh         8543                  mary    2w     FIFO               0,12       0t0    1428938 pipe
python     8544                  mary    2w     FIFO               0,12       0t0    1428938 pipe

[jtratner]

Side note - It would really help if we could somehow stream output from the resolver rather than blocking until the resolver returns (just to make it easier to use verbose to debug where hang is happening) On Sun, Apr 1, 2018 at 5:21 PM MarenstainBear notifications@github.com wrote:

I installed from master at commit 8a67a21 https://github.com/pypa/pipenv/commit/8a67a21d61a2383253fe0dd5e7a8d79d51d30d2d (dated Sat Mar 31 01:13:26 2018 -0400). My issue was not resolved. I saw the same behavior here in version 11.9.1 as I was seeing in 11.9.0.

I did briefly try a different way to debug this problem using strace. The command strace pipenv install tensorflow hung at poll([{fd=5, events=POLLIN}, {fd=7, events=POLLIN}], 2, -1). When I looked up those file descriptors, there wasn't much of interest going on other than confirmation that the system is waiting for I/O.

mary@marvel:/proc/8471/fd

lsof -n -P | grep 1428937 pipenv 8474 mary 5r FIFO 0,12 0t0 1428937 pipe sh 8543 mary 1w FIFO 0,12 0t0 1428937 pipe python 8544 mary 1w FIFO 0,12 0t0 1428937 pipe

mary@marvel:/proc/8471/fd

lsof -n -P | grep 1428938 pipenv 8474 mary 7r FIFO 0,12 0t0 1428938 pipe sh 8543 mary 2w FIFO 0,12 0t0 1428938 pipe python 8544 mary 2w FIFO 0,12 0t0 1428938 pipe

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pypa/pipenv/issues/1816#issuecomment-377828171, or mute the thread https://github.com/notifications/unsubscribe-auth/ABhjq95pb8cZ7BbzrjPxBq3SLeirzFehks5tkW8KgaJpZM4S0kt- .

[roveo]

I'm having the same problem. It hanged indefinitely in the past, but last time it ended with an exception for some reason. Maybe this will help. (Version is 11.9.0)

❯ pipenv install
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
y", line 63, in resolve
    verbose=verbose,
  File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 494, in resolve_deps
    list(resolver.resolve_hashes([result]).items())[0][1]
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 72, in resolve_hashes
    return {ireq: self.repository.get_hashes(ireq) for ireq in ireqs}
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 72, in <dictcomp>
    return {ireq: self.repository.get_hashes(ireq) for ireq in ireqs}
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 276, in get_hashes
    for candidate in matching_candidates
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 276, in <setcomp>
    for candidate in matching_candidates
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 282, in _get_file_hash
    for chunk in iter(lambda: fp.read(8096), b""):
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 282, in <lambda>
    for chunk in iter(lambda: fp.read(8096), b""):
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/requests/packages/urllib3/response.py", line 324, in read
    flush_decoder = True
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/pip9/_vendor/requests/packages/urllib3/response.py", line 237, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pip9._vendor.requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='pypi.python.org', port=443): Read timed out.```

[ghost]

@MarenstainBear Thanks that you're putting so much effort in debugging this issue.

After the installation of the current version of the tensorflow-gpu package failed many times in the same way described, I've managed to complete its installation by pinning it to version 1.5

pipenv install tensorflow-gpu==1.5

Even though it doesn't explain the NumPy and the other packages installation failures, it might have something to do with the fact that since version 1.6, the tensorflow binaries are using AVX instructions. In any case, version 1.5 installs very fast and works just fine.

[MarenstainBear]

@paraschas I tried your tip, and unfortunately it didn't work. Thanks for suggesting something, though! :-)

[techalchemy]

@MarenstainBear this could easily be due to pipenv leaking file descriptors somewhere along the chain and then waiting for them... During installation we usually fork a bunch of subprocesses to handle concurrent downloads and such. What happens if you use pipenv install --sequential? Thanks for bearing with us, do feel free to stop by our slack channel (link is https://pyslackers.com/ => sign up and join #pipenv) if you want to kick more stuff around in real time

[MarenstainBear]

@techalchemy From what I can tell, when it hangs it is never exiting this try block: @techttps://github.com/pypa/pipenv/blob/8a67a21d61a2383253fe0dd5e7a8d79d51d30d2d/pipenv/utils.py#L491-L494

That code is calling back into resolver.py, so I'll see what's happening over there in the resolve_hashes() method.

PS - I did join the slack channel. Thanks for the invite!

[claytonaalves]

I'm facing the same issue here. Python 3.6.4, pipenv 11.9.0. It doesn't hang if I install with --skip-lock.

[MarenstainBear]

@techalchemy @uranusjr

I was able to use ipdb to narrow my issue down considerably...

From the original pypi.py code here, if I add two lines, as shown below, then my pipenv is no longer broken.

1. The first line alt_session = PipSession() creates a vanilla PipSession object
2. And the second line self.session.adapters["https://"] = alt_session.adapters["https://"] grabs the https adapter from that vanilla session and overwrites the PyPIRepository's https adapter. This means it goes from using a pip9._vendor.cachecontrol.adapter.CacheControlAdapter to instead using a pip9._vendor.requests.adapters.HTTPAdapter.

def _get_file_hash(self, location):
    h = hashlib.new(FAVORITE_HASH)
    alt_session = PipSession()
    self.session.adapters["https://"] = alt_session.adapters["https://"]
    with open_local_or_remote_file(location, self.session) as fp:
        for chunk in iter(lambda: fp.read(8096), b""):
            h.update(chunk)
    return ":".join([FAVORITE_HASH, h.hexdigest()])

Any ideas on why this is working for me and what this means about the source of my bug?

[uranusjr]

My guess is it was probably broken because it was trying to fetch too many things at the same time. Switching to a cached adapter reduces the requests it needs (because they are cached) and made it work. This is almost completely a guess (I didn’t look into pip internals and am speaking purely based on my memory to the pip source). If this is correct, however, this would be a very nice fix. Would you care to tiny up the patch into a pull request?

[MarenstainBear]

@uranusjr It's actually the other way around. The original code is using the cached adapter--the cached adapter is the one that is NOT working for me. The regular HTTP adapter does work when I use it instead.

[MarenstainBear]

@uranusjr @techalchemy Now that I think about it, I wonder if there is some kind of corrupt data in my cache. I'll look into how to clear my (pip?) cache to see if that resolves the issue for me.

[uranusjr]

Probably, I don’t know (pip internal is very difficult). pip’s cache is at ~/.cache/pip by default, but maybe there are other caches. I honestly don’t know.

[MarenstainBear]

@uranusjr @techalchemy @jtratner

Holy buckets, Batman! It worked!

mary@marvel:~
> sudo rm ~/.cache/pip* -rf
[sudo] password for mary: 

mary@marvel:~
> mktmpenv 
Using base prefix '/usr'
New python executable in /home/mary/Development/.virtualenvs/tmp-60224356c1829db/bin/python3
Also creating executable in /home/mary/Development/.virtualenvs/tmp-60224356c1829db/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/mary/Development/.virtualenvs/tmp-60224356c1829db/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/mary/Development/.virtualenvs/tmp-60224356c1829db/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/mary/Development/.virtualenvs/tmp-60224356c1829db/bin/preactivate
virtualenvwrapper.user_scripts creating /home/mary/Development/.virtualenvs/tmp-60224356c1829db/bin/postactivate
virtualenvwrapper.user_scripts creating /home/mary/Development/.virtualenvs/tmp-60224356c1829db/bin/get_env_details
This is a temporary environment. It will be deleted when you run 'deactivate'.
(tmp-60224356c1829db) 
mary@marvel:~/Development/.virtualenvs/tmp-60224356c1829db
> pipenv install protobuf
Creating a Pipfile for this project…
Installing protobuf…
Collecting protobuf
  Downloading protobuf-3.5.2.post1-cp35-cp35m-manylinux1_x86_64.whl (6.4MB)
Requirement already satisfied: setuptools in ./lib/python3.5/site-packages (from protobuf)
Collecting six>=1.9 (from protobuf)
  Downloading six-1.11.0-py2.py3-none-any.whl
Installing collected packages: six, protobuf
Successfully installed protobuf-3.5.2.post1 six-1.11.0

Adding protobuf to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (735738)!
Installing dependencies from Pipfile.lock (735738)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 2/2 — 00:00:00
(tmp-60224356c1829db) 
mary@marvel:~/Development/.virtualenvs/tmp-60224356c1829db
> 

Let's see if this solution works for some of the other people reporting the same issue. @roddds @jlhood @roveo @claytonaalves If you're still able to reproduce pipenv hanging indefinitely at "Lock... [packages] dependencies", try clearing your pip (and pipenv?) cache at ~/.cache/.pip and report back on whether that resolved the issue with pipenv hanging. :-)

[uranusjr]

So your prompt says @marvel but you’re quoting Robin…hmm 😏

[techalchemy]

lol... so clearing your pip cache fixed it... life is amusing...

[ghost]

I run across the same issue with the latest Python, pip, and pipenv while installing pendulum and, unfortunately, clearing pip's and pipenv's cache by running `sudo rm ~/.cache/pip* -rf' doesn't solve the problem.

So the issue, at least for me, still persists.

[techalchemy]

@paraschas can you give us the pipfile and the output of python -m pipenv.help

[ghost]

The contents of the Pipfile are:

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pandas = "*"
"h5py" = "*"
Keras = "*"
tensorflow-gpu = "==1.5"
sklearn = "*"
pendulum = "*"

[dev-packages]

[requires] = "3.6"

And the output of python -m pipenv.help is attached:

pipenv.help.txt

Any ideas on how to bypass the issue, even manually, will be greatly appreciated.

[techalchemy]

Can you run pipenv-resolver —debug pendulum

[ghost]

Here it is, not sure of why it fails:

pipenv-resolver_pendulum.txt

[techalchemy]

When debugging it’s helpful if you don’t cut out the command you executed also because I too have no idea why this is failing. Is your locale configured properly etc?