search

pypa / pipenv

Python Development Workflow for Humans.
https://pipenv.pypa.io
MIT License
24.92k stars 1.87k forks source link

`pipenv shell` doesn't play nice with interactive prompts during shell startup #3615

Open ksze opened 5 years ago

ksze commented 5 years ago

Issue description

With pipenv version 2018.11.26 pipenv shell fails to actually activate the virtualenv if an interactive prompt shows up when the shell is entered.

E.g. I use zsh and have oh-my-zsh installed. Periodically, oh-my-zsh will ask me whether I want to update as I enter zsh. If that happens during pipenv shell, the virtualenv's activate script is not sourced.

Expected result

oh-my-zsh's prompt will wait for my input, and then source the activate script.

Actual result

pipenv naïvely "answers" oh-my-zsh's prompt with ". /home/kal/.local/share/virtualenvs/avareference-g9IVq0Y/bin/activate" and then ends up not actually activating the virtualenv.

Steps to replicate

  1. Setup zsh as your default shell;
  2. Install oh-my-zsh the usual way (https://github.com/robbyrussell/oh-my-zsh#basic-installation);
  3. Create any random virtualenv using pipenv;
  4. Force oh-my-zsh to ask for update the next time you launch zsh (how? to be determined); (Alternatively, just insert a random interactive prompt in your .zshrc; e.g. read "your_name?What is your name? ")
  5. Try to enter the virtualenv with pipenv shell;
  6. Watch the activate script sourcing get eaten by oh-my-zsh's update prompt.

The problem seems to stem from pipenv's use of pexpect at https://github.com/pypa/pipenv/blob/master/pipenv/shells.py#L105-L107

I think most Linux/Unix shells (dash, bash, ksh, (t)csh, zsh, even fish) support the -c option. Can we use that instead of sourcing the virtualenv activate script via pexpect?

$ pipenv --support Pipenv version: `'2018.11.26'` Pipenv location: `'/home/kal/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv'` Python location: `'/home/kal/.pyenv/versions/3.7.2/bin/python3.7'` Python installations found: - `3.7.2`: `/home/kal/.pyenv/versions/3.7.2/bin/python3.7m` - `3.7.2`: `/home/kal/.pyenv/versions/3.7.2/bin/python3.7` - `3.7.1`: `/home/kal/.pyenv/versions/3.7.1/bin/python3.7m` - `3.7.1`: `/home/kal/.pyenv/versions/3.7.1/bin/python3.7` - `3.7.0`: `/home/kal/.pyenv/versions/3.7.0/bin/python3.7m` - `3.7.0`: `/home/kal/.pyenv/versions/3.7.0/bin/python3.7` - `3.6.7`: `/usr/bin/python3.6m` - `3.6.7`: `/usr/bin/python3.6` - `3.6.6`: `/home/kal/.pyenv/versions/3.6.6/bin/python3.6m` - `3.6.6`: `/home/kal/.pyenv/versions/3.6.6/bin/python` - `2.7.15`: `/home/kal/.pyenv/versions/2.7.15/bin/python2.7` - `2.7.15rc1`: `/usr/bin/python2.7` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.7.2', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.15.0-46-generic', 'platform_system': 'Linux', 'platform_version': '#49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019', 'python_full_version': '3.7.2', 'python_version': '3.7', 'sys_platform': 'linux'} ``` System environment variables:
$ pipenv --support Pipenv version: `'2018.11.26'` Pipenv location: `'/home/kal/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv'` Python location: `'/home/kal/.pyenv/versions/3.7.2/bin/python3.7'` Python installations found: - `3.7.2`: `/home/kal/.pyenv/versions/3.7.2/bin/python3.7m` - `3.7.2`: `/home/kal/.pyenv/versions/3.7.2/bin/python3.7` - `3.7.1`: `/home/kal/.pyenv/versions/3.7.1/bin/python3.7m` - `3.7.1`: `/home/kal/.pyenv/versions/3.7.1/bin/python3.7` - `3.7.0`: `/home/kal/.pyenv/versions/3.7.0/bin/python3.7m` - `3.7.0`: `/home/kal/.pyenv/versions/3.7.0/bin/python3.7` - `3.6.7`: `/usr/bin/python3.6m` - `3.6.7`: `/usr/bin/python3.6` - `3.6.6`: `/home/kal/.pyenv/versions/3.6.6/bin/python3.6m` - `3.6.6`: `/home/kal/.pyenv/versions/3.6.6/bin/python` - `2.7.15`: `/home/kal/.pyenv/versions/2.7.15/bin/python2.7` - `2.7.15rc1`: `/usr/bin/python2.7` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.7.2', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.15.0-46-generic', 'platform_system': 'Linux', 'platform_version': '#49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019', 'python_full_version': '3.7.2', 'python_version': '3.7', 'sys_platform': 'linux'} ``` System environment variables: - `CLUTTER_IM_MODULE` - `LS_COLORS` - `LC_MEASUREMENT` - `LC_PAPER` - `LC_MONETARY` - `XDG_MENU_PREFIX` - `LANG` - `LESS` - `DISPLAY` - `PYENV_ROOT` - `OLDPWD` - `_ZSH_TMUX_FIXED_CONFIG` - `GNOME_SHELL_SESSION_MODE` - `COLORTERM` - `USERNAME` - `PYENV_VIRTUALENV_INIT` - `PYENV_HOOK_PATH` - `XDG_VTNR` - `ZSH` - `SSH_AUTH_SOCK` - `MANDATORY_PATH` - `S_COLORS` - `LC_NAME` - `XDG_SESSION_ID` - `USER` - `PYENV_DIR` - `PAGER` - `LSCOLORS` - `DESKTOP_SESSION` - `QT4_IM_MODULE` - `TEXTDOMAINDIR` - `GNOME_TERMINAL_SCREEN` - `DEFAULTS_PATH` - `PWD` - `HOME` - `LC_CTYPE` - `TEXTDOMAIN` - `SSH_AGENT_PID` - `PYENV_VERSION` - `QT_ACCESSIBILITY` - `TMUX` - `XDG_SESSION_TYPE` - `XDG_DATA_DIRS` - `XDG_SESSION_DESKTOP` - `LC_ADDRESS` - `GJS_DEBUG_OUTPUT` - `LC_NUMERIC` - `ZSH_TMUX_TERM` - `GTK_MODULES` - `PAPERSIZE` - `WINDOWPATH` - `VTE_VERSION` - `TERM` - `SHELL` - `QT_IM_MODULE` - `XMODIFIERS` - `IM_CONFIG_PHASE` - `XDG_CURRENT_DESKTOP` - `GPG_AGENT_INFO` - `GNOME_TERMINAL_SERVICE` - `TMUX_PANE` - `XDG_SEAT` - `SHLVL` - `PYENV_SHELL` - `LANGUAGE` - `LC_TELEPHONE` - `GDMSESSION` - `GNOME_DESKTOP_SESSION_ID` - `LOGNAME` - `DBUS_SESSION_BUS_ADDRESS` - `XDG_RUNTIME_DIR` - `XAUTHORITY` - `XDG_CONFIG_DIRS` - `PATH` - `LC_IDENTIFICATION` - `GJS_DEBUG_TOPICS` - `SESSION_MANAGER` - `GTK_IM_MODULE` - `LC_TIME` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/kal/.pyenv/versions/3.7.2/bin:/home/kal/.pyenv/libexec:/home/kal/.pyenv/plugins/python-build/bin:/home/kal/.pyenv/plugins/pyenv-virtualenv/bin:/home/kal/.pyenv/plugins/pyenv-update/bin:/home/kal/.pyenv/plugins/pyenv-installer/bin:/home/kal/.pyenv/plugin s/pyenv-doctor/bin:/home/kal/.pyenv/shims:/home/kal/.local/bin:/home/kal/.cargo/bin:/home/kal/.pyenv/plugins/pyenv-virtualenv/shims:/home/kal/.pyenv/shims:/home/kal/.pyenv/bin:/home/kal/.pyenv/shims:/home/kal/.local/bin:/home/kal/.cargo/bin:/home/kal/.pyenv/plugins/pyenv- virtualenv/shims:/home/kal/.pyenv/shims:/home/kal/.pyenv/bin:/home/kal/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin` - `SHELL`: `/usr/bin/zsh` - `LANG`: `en_GB.UTF-8` - `PWD`: `/home/kal/.oh-my-zsh` ---------------------------
techalchemy commented 5 years ago

I'm fairly sure there is a good reason why we don't do that but @uranusjr will likely know the specifics

uranusjr commented 5 years ago

Try if the --fancy option works better. I guess technically it is possible to use -c, but someone will need to research on the compatibility across shells and versions.

A list of currently supported shells can be found here: https://github.com/sarugaku/shellingham/blob/master/src/shellingham/_core.py

(And honestly I hate pipenv shell so much. Can everyone just always use pipenv run instead?)

ksze commented 5 years ago

I just tested pipenv shell vs pipenv shell --fancy.

There are slight differences in the resulting environments.

I've captured the environments by doing, e.g.:

$ pipenv shell
$ env | sort > ~/pipenv_shell_env
$ exit
$ pipenv shell --fancy
$ env | sort > ~/pipenv_shell_fancy_env
$ exit
$ diff ~/pipenv_shell_env ~/pipenv_shell_fancy_env
41c41
< PATH=/home/kal/.local/share/virtualenvs/abp-YM2LQhoW/bin:/home/kal/.pyenv/plugins/pyenv-virtualenv/shims:/home/kal/.pyenv/shims:/home/kal/.pyenv/versions/3.7.2/bin:/home/kal/.pyenv/libexec:/home/kal/.pyenv/plugins/python-build/bin:/home/kal/.pyenv/plugins/pyenv-virtualenv/bin:/home/kal/.pyenv/plugins/pyenv-update/bin:/home/kal/.pyenv/plugins/pyenv-installer/bin:/home/kal/.pyenv/plugins/pyenv-doctor/bin:/home/kal/.pyenv/shims:/home/kal/.local/bin:/home/kal/.cargo/bin:/home/kal/.pyenv/plugins/pyenv-virtualenv/shims:/home/kal/.pyenv/shims:/home/kal/.pyenv/bin:/home/kal/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
---
> PATH=/home/kal/.pyenv/plugins/pyenv-virtualenv/shims:/home/kal/.pyenv/shims:/home/kal/.local/share/virtualenvs/abp-YM2LQhoW/bin:/home/kal/.pyenv/versions/3.7.2/bin:/home/kal/.pyenv/libexec:/home/kal/.pyenv/plugins/python-build/bin:/home/kal/.pyenv/plugins/pyenv-virtualenv/bin:/home/kal/.pyenv/plugins/pyenv-update/bin:/home/kal/.pyenv/plugins/pyenv-installer/bin:/home/kal/.pyenv/plugins/pyenv-doctor/bin:/home/kal/.pyenv/shims:/home/kal/.local/bin:/home/kal/.cargo/bin:/home/kal/.pyenv/plugins/pyenv-virtualenv/shims:/home/kal/.pyenv/shims:/home/kal/.pyenv/bin:/home/kal/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
45d44
< PS1=(abp-YM2LQhoW) %f%b%k$(build_left_prompt)

Note how --fancy places the virtualenv's bin directory after the pyenv shim directories, which I believe is wrong. It also doesn't set $PS1, though that's just a minor cosmetic problem (and it's ignored by the powerlevel9k theme for zsh that I use anyway).

uranusjr commented 5 years ago

--fancy and the default (compat) shell are invoked differently. Fancy mode injects the environment first, and then launches the shell (which is cleaner), while compat mode tries to be smart, and inject the environment after launching the shell.

The nature how fancy mode works means that it requires the user to configure the shell in a very strict best-practice way. In you case, you probably have pyenv init - (or something similar) in your zshrc, which unconditionally prepends pyenv paths to PATH, resulting in the “incorrect” result (in quotes because yeah, that’s not what you want, but Pipenv has no way to help it).

Going back to the original issue. To switch to -c, someone would need to research on the compatibility across shells and versions. The implementation should come easily after the hard work is done.