pypa / pipenv

Python Development Workflow for Humans.
https://pipenv.pypa.io
MIT License
24.81k stars 1.86k forks source link

Unable to install pycountry #3245

Closed studybuffalo closed 5 years ago

studybuffalo commented 5 years ago

Issue description

When trying to install pycountry (either via Pipfile or pipenv install pycountry), pipenv becomes stuck on the installing stage. There are unfortunately no errors or outputs that I have been able to generate to narrow this issue down.

Expected result

The pycountry package to be installed.

Actual result

  $ pipenv install pycountry --verbose
  Installing pycountry…
  [    ] Installing...Installing 'pycountry'
  $ ['C:\\Users\\Studybuffalo\\.virtualenvs\\test-23MAWioC\\Scripts\\pip.exe', 'install', '--verbose', '--upgrade', 'pycountry', '-i', 'https://pypi.org/simple']
  [  ==] Installing...

The "Installing" output continues indefinitely (have left it for over an hour). When cancelling the install the only output is: FAIL

Steps to replicate

Using Windows 10, Python 3.7.1, and pipenv 2018.11.14:

  1. pipenv install pycountry

Other Details

Pipenv installed via pip (pip install pipenv==2018.11.14)

Output from pipenv --support

$ pipenv --support Pipenv version: `'2018.11.14'` Pipenv location: `'c:\\program files (x86)\\python37\\lib\\site-packages\\pipenv'` Python location: `'c:\\program files (x86)\\python37\\python.exe'` Python installations found: - `3.7.1`: `C:\Program Files (x86)\Python37\python.exe` - `3.6.7`: `C:\Program Files (x86)\Python36\python.exe` - `3.5`: `C:\Program Files (x86)\Python35\python.exe` - `3.4`: `C:\Program Files (x86)\Python34\python.exe` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.7.1', 'os_name': 'nt', 'platform_machine': 'AMD64', 'platform_python_implementation': 'CPython', 'platform_release': '10', 'platform_system': 'Windows', 'platform_version': '10.0.17134', 'python_full_version': '3.7.1', 'python_version': '3.7', 'sys_platform': 'win32'} ``` System environment variables: - `ALLUSERSPROFILE` - `APPDATA` - `ASL.LOG` - `COMMONPROGRAMFILES` - `COMMONPROGRAMFILES(X86)` - `COMMONPROGRAMW6432` - `COMPUTERNAME` - `COMSPEC` - `DRIVERDATA` - `FP_NO_HOST_CHECK` - `HOMEDRIVE` - `HOMEPATH` - `LOCALAPPDATA` - `LOGONSERVER` - `NUMBER_OF_PROCESSORS` - `ONEDRIVE` - `OS` - `PATH` - `PATHEXT` - `PIPENV_INSTALL_TIMEOUT` - `PROCESSOR_ARCHITECTURE` - `PROCESSOR_IDENTIFIER` - `PROCESSOR_LEVEL` - `PROCESSOR_REVISION` - `PROGRAMDATA` - `PROGRAMFILES` - `PROGRAMFILES(X86)` - `PROGRAMW6432` - `PSMODULEPATH` - `PUBLIC` - `SESSIONNAME` - `SYSTEMDRIVE` - `SYSTEMROOT` - `TEMP` - `TESSDATA_PREFIX` - `TMP` - `USERDOMAIN` - `USERDOMAIN_ROAMINGPROFILE` - `USERNAME` - `USERPROFILE` - `VBOX_MSI_INSTALL_PATH` - `VS100COMNTOOLS` - `VS110COMNTOOLS` - `VS120COMNTOOLS` - `VS140COMNTOOLS` - `WINDIR` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv?specific environment variables: - `PIPENV_INSTALL_TIMEOUT`: `60` Debug?specific environment variables: - `PATH`: `C:\Program Files (x86)\Python37\Scripts\;C:\Program Files (x86)\Python37\;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Bitvise SSH Client;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\dotnet\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\PuTTY\;C:\Program Files\Git\cmd;C:\Program Files (x86)\Tesseract-OCR\;C:\Program Files\nodejs\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\gs\gs9.19\bin;C:\Program Files\gs\gs9.19\lib;C:\WINDOWS\system32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Users\Studybuffalo\AppData\Local\Microsoft\WindowsApps;E:\My Documents\GitHub\config\;C:\Program Files\Microsoft VS Code\bin;C:\Users\Studybuffalo\AppData\Roaming\npm;C:\Users\Studybuffalo\AppData\Local\GitHubDesktop\bin;C:\Users\Studybuffalo\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Studybuffalo\AppData\Local\Programs\Python\Python37\;C:\Users\Studybuffalo\AppData\Local\Programs\Python\Python37\Scripts;` --------------------------- Contents of `Pipfile` ('E:\\My Documents\\GitHub\\test\\Pipfile'): ```toml [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev-packages] [packages] django = "*" [requires] python_version = "3.7" ``` Contents of `Pipfile.lock` ('E:\\My Documents\\GitHub\\test\\Pipfile.lock'): ```json { "_meta": { "hash": { "sha256": "627ef89f247ecee27e9ef0dabe116108d09c47abf171c900a8817befa64f9dd2" }, "pipfile-spec": 6, "requires": { "python_version": "3.7" }, "sources": [ { "name": "pypi", "url": "https://pypi.org/simple", "verify_ssl": true } ] }, "default": { "django": { "hashes": [ "sha256:1ffab268ada3d5684c05ba7ce776eaeedef360712358d6a6b340ae9f16486916", "sha256:dd46d87af4c1bf54f4c926c3cfa41dc2b5c15782f15e4329752ce65f5dad1c37" ], "index": "pypi", "version": "==2.1.3" }, "pytz": { "hashes": [ "sha256:8e0f8568c118d3077b46be7d654cc8167fa916092e28320cde048e54bfc9f1e6" ], "version": "==2018.7" } }, "develop": {} } ```

Attempted Fixes

The following attempts did not result in any changes to the above:

  1. Installing in a fresh virtual environment.
  2. Clearing the cache when installing (with the --clear option).
  3. Downgrading pipenv version to 2018.10.13 and 2018.10.09.
  4. Reducing the PIPENV_INSTALL_TIMEOUT variable to < 60 seconds to trigger a timeout.
  5. Running installation with Administrator rights.
  6. Verified installations work by installing other packages (django in the above output for example)

The following attempts had varying success:

  1. Installing pycountry via pip (pip install pycountry) in pipenv shell works. The Pipfile was not updated, but pycountry was listed with pipenv graph.
  2. Installing into an Ubuntu 16.04 server worked as expected. pipenv --support output as follows:
$ pipenv --support Pipenv version: `'2018.11.14'` Pipenv location: `'/usr/local/lib/python3.6/dist-packages/pipenv'` Python location: `'/usr/bin/python3.6'` Python installations found: - `3.6.7`: `/usr/bin/python3.6m` - `3.6.7`: `/usr/bin/python3.6` - `3.5.2`: `/usr/bin/python3.5` - `3.5.2`: `/usr/bin/python3.5m` - `2.7.12`: `/usr/bin/python2` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.6.7', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.4.0-139-generic', 'platform_system': 'Linux', 'platform_version': '#165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018', 'python_full_version': '3.6.7', 'python_version': '3.6', 'sys_platform': 'linux'} ``` System environment variables: - `XDG_SESSION_ID` - `TERM` - `SHELL` - `SSH_CLIENT` - `SSH_TTY` - `USER` - `LS_COLORS` - `MAIL` - `PATH` - `PWD` - `LANG` - `SHLVL` - `HOME` - `LANGUAGE` - `LOGNAME` - `XDG_DATA_DIRS` - `SSH_CONNECTION` - `LESSOPEN` - `XDG_RUNTIME_DIR` - `LESSCLOSE` - `_` - `OLDPWD` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/studybuffalo/bin:/home/studybuffalo/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin` - `SHELL`: `/bin/bash` - `LANG`: `en_CA.UTF-8` - `PWD`: `/home/studybuffalo/test` --------------------------- Contents of `Pipfile` ('/home/studybuffalo/test/Pipfile'): ```toml [[source]] url = "https://pypi.org/simple" verify_ssl = true name = "pypi" [packages] pycountry = "*" [dev-packages] [requires] python_version = "3.6" ``` Contents of `Pipfile.lock` ('/home/studybuffalo/test/Pipfile.lock'): ```json { "_meta": { "hash": { "sha256": "792a57e440e7887d9fe04b0305cc0bc953779d266aa9204b4075626a7eee844f" }, "pipfile-spec": 6, "requires": { "python_version": "3.6" }, "sources": [ { "name": "pypi", "url": "https://pypi.org/simple", "verify_ssl": true } ] }, "default": { "pycountry": { "hashes": [ "sha256:195d8174fd6f98f45622d2885826091fc8b2168184422dac574311073ab8386c", "sha256:7f2aa2529c60f6575af3cd644688b201b97016822ce0ce1c4bcc0f7d08900997", "sha256:ea38f839e719be54dfae1b089cbf1917add6ff8f43a55533d8c7c5799656a848" ], "index": "pypi", "version": "==18.5.26" } }, "develop": {} } ```
techalchemy commented 5 years ago

Well you have PIPENV_INSTALL_TIMEOUT at 60 so the 1 hour thing makes sense. You can try to debug by running pipenv lock --verbose and providing the output

studybuffalo commented 5 years ago

Tried it again both with no timeout (left for 2 hours) and then again with a 1 minute timeout. Both times it remained on [====] Locking... stage. No lock file was generated and I eventually had to break the loop (ctrl + c did not work). One of these times actually did yield a traceback (not sure if it is related to this issue or just the fact I had to force it to terminate):

PS E:\My Documents\GitHub\test> pipenv lock --verbose
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
FAIL
Traceback (most recent call last):
  File "c:\program files (x86)\python37\lib\subprocess.py", line 984, in wait
    return self._wait(timeout=timeout)
  File "c:\program files (x86)\python37\lib\subprocess.py", line 1224, in _wait
    timeout_millis)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\yaspin\signal_handlers.py", line 23, in default_handler
    sys.exit(0)
SystemExit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\program files (x86)\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\program files (x86)\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Program Files (x86)\Python37\Scripts\pipenv.exe\__main__.py", line 9, in <module>
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\core.py", line 717, in main
    rv = self.invoke(ctx)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\click\decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\cli\command.py", line 325, in lock
    pypi_mirror=state.pypi_mirror,
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\core.py", line 1047, in do_lock
    lockfile=lockfile
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\utils.py", line 620, in venv_resolve_deps
    c = resolve(cmd, sp)
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\utils.py", line 518, in resolve
    c.block()
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\delegator.py", line 261, in block
    self.subprocess.wait()
  File "c:\program files (x86)\python37\lib\site-packages\pipenv\vendor\pexpect\popen_spawn.py", line 161, in wait
    status = self.proc.wait()
  File "c:\program files (x86)\python37\lib\subprocess.py", line 984, in wait
    return self._wait(timeout=timeout)
TypeError: 'int' object is not callable

Steps to Replicate

  1. Created a pipfile with the following:
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pycountry = "*"

[requires]
python_version = "3.7"
  1. $ pipenv lock --verbose
$ pipenv --support Pipenv version: `'2018.11.14'` Pipenv location: `'c:\\program files (x86)\\python37\\lib\\site-packages\\pipenv'` Python location: `'c:\\program files (x86)\\python37\\python.exe'` Python installations found: - `3.7.1`: `C:\Program Files (x86)\Python37\python.exe` - `3.6.7`: `C:\Program Files (x86)\Python36\python.exe` - `3.5`: `C:\Program Files (x86)\Python35\python.exe` - `3.4`: `C:\Program Files (x86)\Python34\python.exe` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.7.1', 'os_name': 'nt', 'platform_machine': 'AMD64', 'platform_python_implementation': 'CPython', 'platform_release': '10', 'platform_system': 'Windows', 'platform_version': '10.0.17134', 'python_full_version': '3.7.1', 'python_version': '3.7', 'sys_platform': 'win32'} ``` System environment variables: - `ALLUSERSPROFILE` - `APPDATA` - `ASL.LOG` - `COMMONPROGRAMFILES` - `COMMONPROGRAMFILES(X86)` - `COMMONPROGRAMW6432` - `COMPUTERNAME` - `COMSPEC` - `DRIVERDATA` - `FP_NO_HOST_CHECK` - `HOMEDRIVE` - `HOMEPATH` - `LOCALAPPDATA` - `LOGONSERVER` - `NUMBER_OF_PROCESSORS` - `ONEDRIVE` - `OS` - `PATH` - `PATHEXT` - `PIPENV_INSTALL_TIMEOUT` - `PROCESSOR_ARCHITECTURE` - `PROCESSOR_IDENTIFIER` - `PROCESSOR_LEVEL` - `PROCESSOR_REVISION` - `PROGRAMDATA` - `PROGRAMFILES` - `PROGRAMFILES(X86)` - `PROGRAMW6432` - `PSMODULEPATH` - `PUBLIC` - `SYSTEMDRIVE` - `SYSTEMROOT` - `TEMP` - `TESSDATA_PREFIX` - `TMP` - `USERDOMAIN` - `USERDOMAIN_ROAMINGPROFILE` - `USERNAME` - `USERPROFILE` - `VBOX_MSI_INSTALL_PATH` - `VS100COMNTOOLS` - `VS110COMNTOOLS` - `VS120COMNTOOLS` - `VS140COMNTOOLS` - `WINDIR` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv?specific environment variables: - `PIPENV_INSTALL_TIMEOUT`: `1` Debug?specific environment variables: - `PATH`: `C:\Program Files (x86)\Python37\Scripts\;C:\Program Files (x86)\Python37\;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Bitvise SSH Client;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\dotnet\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\PuTTY\;C:\Program Files\Git\cmd;C:\Program Files (x86)\Tesseract-OCR\;C:\Program Files\nodejs\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\gs\gs9.19\bin;C:\Program Files\gs\gs9.19\lib;C:\WINDOWS\system32\WindowsPowerShell\v1.0\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Users\studybuffalo\AppData\Local\Microsoft\WindowsApps;E:\My Documents\GitHub\config\;C:\Program Files\Microsoft VS Code\bin;C:\Users\studybuffalo\AppData\Roaming\npm;C:\Users\studybuffalo\AppData\Local\GitHubDesktop\bin;C:\Users\studybuffalo\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\studybuffalo\AppData\Local\Programs\Python\Python37\;C:\Users\studybuffalo\AppData\Local\Programs\Python\Python37\Scripts` --------------------------- Contents of `Pipfile` ('E:\\My Documents\\GitHub\\test\\Pipfile'): ```toml [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev-packages] [packages] pycountry = "*" [requires] python_version = "3.7" ```
u0m3 commented 5 years ago

I seem to be running into a similar issue with pipenv 2018.11.26 on Windows 10 Version 1803 (OS Build 17134.4037). Only in my case, the traceback is different somewhat:

radua@DESKTOP-2GT09E2 C:\Git\hackthebox-scripts
> py -3.7-64 -m pipenv install "asyncssh[bcrypt,libnacl,pyOpenSSL,pypiwin32]"
Installing asyncssh[bcrypt,libnacl,pyOpenSSL,pypiwin32]…
Adding asyncssh to Pipfile's [packages]…
Installation Succeeded
Pipfile.lock (e091ee) out of date, updating to (a65489)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Traceback (most recent call last):
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\pexpect\expect.py", line 109, in expect_loop
    return self.timeout()
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\pexpect\expect.py", line 82, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x0000022A3D7AAF28>
searcher: searcher_re:
    0: re.compile('\n')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python37_x86_64\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python37_x86_64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\__main__.py", line 4, in <module>
    cli()
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\core.py", line 717, in main
    rv = self.invoke(ctx)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\click\decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\cli\command.py", line 254, in install
    editable_packages=state.installstate.editables,
  File "C:\Python37_x86_64\lib\site-packages\pipenv\core.py", line 1992, in do_install
    skip_lock=skip_lock,
  File "C:\Python37_x86_64\lib\site-packages\pipenv\core.py", line 1221, in do_init
    pypi_mirror=pypi_mirror,
  File "C:\Python37_x86_64\lib\site-packages\pipenv\core.py", line 1068, in do_lock
    lockfile=lockfile
  File "C:\Python37_x86_64\lib\site-packages\pipenv\utils.py", line 649, in venv_resolve_deps
    c = resolve(cmd, sp)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\utils.py", line 517, in resolve
    result = c.expect(u"\n", timeout=environments.PIPENV_INSTALL_TIMEOUT)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\delegator.py", line 215, in expect
    self.subprocess.expect(pattern=pattern, timeout=timeout)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\pexpect\spawnbase.py", line 341, in expect
    timeout, searchwindowsize, async_)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\pexpect\spawnbase.py", line 369, in expect_list
    return exp.expect_loop(timeout)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\pexpect\expect.py", line 119, in expect_loop
    return self.timeout(e)
  File "C:\Python37_x86_64\lib\site-packages\pipenv\vendor\pexpect\expect.py", line 82, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x0000022A3D7AAF28>
searcher: searcher_re:
    0: re.compile('\n')
<pexpect.popen_spawn.PopenSpawn object at 0x0000022A3D7AAF28>
searcher: searcher_re:
    0: re.compile('\n')

From what I have seen, running py -3.7-64 -m pipenv install "asyncssh[bcrypt,libnacl,pyOpenSSL,pypiwin32]" --verbose launches C:\Python37_x86_64\python.exe -m pipenv install "asyncssh[bcrypt,libnacl,pyOpenSSL,pypiwin32]" --verbose which in turn launches C:\Users\radua\pyvenvs\hackthebox-scripts-rhykpVsX\Scripts\python.exe C:/Python37_x86_64/lib/site-packages/pipenv/resolver.py I attempted to add import pdb; pdb.set_trace() to C:/Python37_x86_64/lib/site-packages/pipenv/resolver.py right after the if __name__ == "__main__": line, so I could force it to break and allow me to step through the code to see what does wrong, but I get the following error:

radua@DESKTOP-2GT09E2 C:\Git\hackthebox-scripts
> py -3.7-64 -m pipenv install "asyncssh[bcrypt,libnacl,pyOpenSSL,pypiwin32]" --verbose
Installing asyncssh[bcrypt,libnacl,pyOpenSSL,pypiwin32]…
[    ] Installing...Installing 'asyncssh'
$ ['C:\\Users\\radua\\pyvenvs\\hackthebox-scripts-rhykpVsX\\Scripts\\pip.exe', 'install', '--verbose', '--upgrade', 'asyncssh[bcrypt,libnacl,pyopenssl,pypiwin32]', '-i', 'https://pypi.org/simple']
Adding asyncssh to Pipfile's [packages]…
Installation Succeeded
Pipfile.lock (e091ee) out of date, updating to (a65489)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Locking Failed!
> c:\python37_x86_64\lib\site-packages\pipenv\resolver.py(124)<module>()
-> _patch_path()
(Pdb)
Traceback (most recent call last):
  File "C:/Python37_x86_64/lib/site-packages/pipenv/resolver.py", line 124, in <module>
    _patch_path()
  File "C:/Python37_x86_64/lib/site-packages/pipenv/resolver.py", line 124, in <module>
    _patch_path()
  File "C:\Python37_x86_64\Lib\bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)
  File "C:\Python37_x86_64\Lib\bdb.py", line 113, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit
-> _patch_path()
(Pdb)
Traceback (most recent call last):
  File "C:/Python37_x86_64/lib/site-packages/pipenv/resolver.py", line 124, in <module>
    _patch_path()
  File "C:/Python37_x86_64/lib/site-packages/pipenv/resolver.py", line 124, in <module>
    _patch_path()
  File "C:\Python37_x86_64\Lib\bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)
  File "C:\Python37_x86_64\Lib\bdb.py", line 113, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit
$ pipenv --support Pipenv version: `'2018.11.26'` Pipenv location: `'C:\\Python37_x86_64\\lib\\site-packages\\pipenv'` Python location: `'C:\\Python37_x86_64\\python.exe'` Python installations found: - `3.7.1`: `C:\Python37_x86_64\python.exe` - `3.7.1`: `C:\Python37_x86_32\python.exe` - `3.6.6`: `C:\Python36_x86_64\python.exe` - `3.6.6`: `C:\Python36_x86_32\python.exe` - `2.7`: `C:\Python27_x86_32\python.exe` - `2.7`: `C:\Python27_x86_64\python.exe` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.7.1', 'os_name': 'nt', 'platform_machine': 'AMD64', 'platform_python_implementation': 'CPython', 'platform_release': '10', 'platform_system': 'Windows', 'platform_version': '10.0.17134', 'python_full_version': '3.7.1', 'python_version': '3.7', 'sys_platform': 'win32'} ``` System environment variables: - `ALLUSERSPROFILE` - `ANSICON` - `ANSICON_DEF` - `APPDATA` - `CHOCOLATEYINSTALL` - `CHOCOLATEYLASTPATHUPDATE` - `COMMONPROGRAMFILES` - `COMMONPROGRAMFILES(X86)` - `COMMONPROGRAMW6432` - `COMPUTERNAME` - `COMSPEC` - `CONEMUANSI` - `CONEMUANSILOG` - `CONEMUARGS` - `CONEMUARGS2` - `CONEMUBACKHWND` - `CONEMUBASEDIR` - `CONEMUBUILD` - `CONEMUCONFIG` - `CONEMUDIR` - `CONEMUDRAWHWND` - `CONEMUDRIVE` - `CONEMUHOOKS` - `CONEMUHWND` - `CONEMUPALETTE` - `CONEMUPID` - `CONEMUPROMPT0` - `CONEMUPROMPT1` - `CONEMUPROMPT2` - `CONEMUPROMPT3` - `CONEMUSERVERPID` - `CONEMUTASK` - `CONEMUWORKDIR` - `CONEMUWORKDRIVE` - `DOCKER_TOOLBOX_INSTALL_PATH` - `DOKANLIBRARY1` - `DOKANLIBRARY1_LIBRARYPATH_X64` - `DOKANLIBRARY1_LIBRARYPATH_X86` - `DRIVERDATA` - `ESET_OPTIONS` - `GPU_FORCE_64BIT_PTR` - `GPU_MAX_ALLOC_PERCENT` - `GPU_MAX_HEAP_SIZE` - `GPU_SINGLE_ALLOC_PERCENT` - `GPU_USE_SYNC_OBJECTS` - `HOMEDRIVE` - `HOMEPATH` - `LANG` - `LANGUAGE` - `LC_ADDRESS` - `LC_ALL` - `LC_COLLATE` - `LC_IDENTIFICATION` - `LC_MEASUREMENT` - `LC_MESSAGES` - `LC_MONETARY` - `LC_NAME` - `LC_NUMERIC` - `LC_PAPER` - `LC_TELEPHONE` - `LC_TIME` - `LC_TYPE` - `LOCALAPPDATA` - `LOGONSERVER` - `NUMBER_OF_PROCESSORS` - `NVM_HOME` - `NVM_SYMLINK` - `ONEDRIVE` - `OS` - `PATH` - `PATHEXT` - `PROCESSOR_ARCHITECTURE` - `PROCESSOR_IDENTIFIER` - `PROCESSOR_LEVEL` - `PROCESSOR_REVISION` - `PROGRAMDATA` - `PROGRAMFILES` - `PROGRAMFILES(X86)` - `PROGRAMW6432` - `PROMPT` - `PSMODULEPATH` - `PUBLIC` - `RG_GPU_FRAMEWORK_ENGINE_RESOURCEDIR` - `SESSIONNAME` - `SW_SIM_HYDRA` - `SW_SIM_MPIT` - `SW_SIM_TEMP` - `SYSTEMDRIVE` - `SYSTEMROOT` - `TEMP` - `TMP` - `USERDOMAIN` - `USERDOMAIN_ROAMINGPROFILE` - `USERNAME` - `USERPROFILE` - `VBOX_MSI_INSTALL_PATH` - `VISUALKERNEL_DIR` - `VS110COMNTOOLS` - `VS120COMNTOOLS` - `VS140COMNTOOLS` - `VSSDK140INSTALL` - `WINDIR` - `WORKON_HOME` - `_NT_SYMBOL_PATH` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `C:\Program Files\ConEmu\ConEmu\Scripts;C:\Program Files\ConEmu;C:\Program Files\ConEmu\ConEmu;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\Windows;C:\Windows\System32;C:\Windows\System32\wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\ProgramData\chocolatey;C:\ProgramData\chocolatey\bin;C:\Python27_x86_32;C:\Python27_x86_32\Scripts;C:\Program Files (x86)\Nmap;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\AMD\ATI.ACE\Core-Static;C:\Program Files\PowerShell\6.0.0\;C:\Program Files\PuTTY\;C:\Program Files\010 Editor;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\ATI Technologies\ATI.ACE\Core-Static;C:\HashiCorp\Vagrant\bin;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files (x86)\Gpg4win\..\GnuPG\bin;C:\Strawberry_Perl_x86_64\c\bin;C:\Strawberry_Perl_x86_64\perl\site\bin;C:\Strawberry_Perl_x86_64\perl\bin;C:\Program Files\Git\cmd;C:\Winginx\php71;C:\Users\radua\AppData\Local\Microsoft\WindowsApps;C:\Users\radua\AppData\Roaming\npm;C:\Program Files (x86)\Nmap;C:\Program Files\Docker Toolbox;C:\Users\radua\AppData\Roaming\nvm;C:\Program Files\nodejs;C:\Users\radua\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\radua\AppData\Local\Programs\Microsoft VS Code Insiders\bin;C:\Users\radua\AppData\Local\Keybase\` - `LANG`: `en_US.UTF-8` --------------------------- Contents of `Pipfile` ('C:\\Git\\hackthebox-scripts\\Pipfile'): ```toml [[source]] url = "https://pypi.org/simple" verify_ssl = true name = "pypi" [packages] asyncssh = {extras = ["bcrypt", "libnacl", "pyopenssl", "pypiwin32"],version = "*"} [dev-packages] [requires] python_version = "3.7" ``` Contents of `Pipfile.lock` ('C:\\Git\\hackthebox-scripts\\Pipfile.lock'): ```json { "_meta": { "hash": { "sha256": "7e7ef69da7248742e869378f8421880cf8f0017f96d94d086813baa518a65489" }, "pipfile-spec": 6, "requires": { "python_version": "3.7" }, "sources": [ { "name": "pypi", "url": "https://pypi.org/simple", "verify_ssl": true } ] }, "default": {}, "develop": {} } ```
studybuffalo commented 5 years ago

@u0m3 gave me the idea to modify to modify my local pipenv version with debug code to try and track down where the issue is occurring and I think I have been able to determine where my issue is occurring.

It seems my pipenv locks up when getting to Command.block (in pipenv/vendor/delegator.py)

I replaced the normal block method with the code below to write some output to disk (printing to console was spotty because at some point stdout is being captured).

def block(self):
    """Blocks until process is complete."""
    if self._uses_subprocess:
        # consume stdout and stderr
        if self.blocking:
            try:
                # Record the start of the subprocess method
                with open('E:\\Dropbox\\test.txt', 'a') as doc:
                    from datetime import datetime
                    doc.write('{}: {}\n'.format(datetime.now(), 'block start'))
                stdout, stderr = self.subprocess.communicate()
                self.__out = stdout
                self.__err = stderr
                # Record the completion of the subprocess method
                with open('E:\\Dropbox\\test.txt', 'a') as doc:
                    from datetime import datetime
                    doc.write('{}: {}\n'.format(datetime.now(), 'block end'))
            except ValueError:
                pass  # Don't read from finished subprocesses.
        else:
            self.subprocess.stdin.close()
            self.std_out.close()
            self.std_err.close()
            self.subprocess.wait()
    else:
        self.subprocess.sendeof()
        try:
            self.subprocess.wait()
        finally:
            if self.subprocess.proc.stdout:
                self.subprocess.proc.stdout.close()

Trying pipenv install pycountry --verbose --clear results in:

2018-12-01 09:16:49.355015: block start

with Pipenv never finishing to install.

Trying a package that works (e.g. pipenv install requests--verbose --clear) results in:

2018-12-01 09:16:41.222944: block start
2018-12-01 09:16:41.932091: block end

Unfortunately from this point Pipenv is using the subprocess module (which then uses the threading module) and I don't have much experience with these modules to understand where the next issue is occurring. If I get some more time in the near future I will try and dig through them to see if I can narrow this down more.

techalchemy commented 5 years ago

Modifying subprocess is not an option, so if your bug requires that to fix you’ll need to file it against the python issue tracker. But it’s more likely that you can modify the environment variable PIPENV_TIMEOUT and set it to 900 and your issue will go away

cambot commented 5 years ago

I understand why modifying a subprocess would not be an option, but what about catching the pexpect.exceptions.TIMEOUT error and bubbling up a more helpful error message (and possibly hints for dealing with it)

techalchemy commented 5 years ago

This was resolved by #3298 (or possibly another pr that merged a fix)

sorry for the issue!

studybuffalo commented 5 years ago

Thanks for that - I will give it a test in the near future!

cambot commented 5 years ago

@techalchemy , where can people go to find information about when the next expected release will be? thx. :)

techalchemy commented 5 years ago

Right here! Catching up on issues and merge requests and then we will release