Virtual enivornment name length causes package setup failures: [Errno 36] File name too long

[badp]

Steps to replicate

cd ~
mkdir i_am_a_very_nice_project_name_and_this_is_fine
pipenv install apache-libcloud

Expected result

apache-libcloud is installed.

Actual result

Could not install packages due to an EnvironmentError: [Errno 36] File name too long: '/home/redacted/.local/share/virtualenvs/i_am_a_very_nice_project_name_and_this_is_-0RLXc4tJ/lib/python3.6/site-packages/libcloud/test/compute/fixtures/azure_arm/_subscriptions_99999999_9999_9999_9999_999999999999_resourceGroups_000000_providers_Microsoft_Compute_virtualMachines_test_node_1_InstanceView.json'

$ python -m pipenv.help output

Pipenv version: `'2018.05.18'` Pipenv location: `'/usr/local/lib/python3.6/dist-packages/pipenv'` Python location: `'/usr/bin/python3'` Other Python installations in `PATH`: - `2.7`: `/usr/bin/python2.7` - `2.7`: `/usr/bin/python2.7` - `3.6`: `/usr/bin/python3.6m` - `3.6`: `/usr/bin/python3.6` - `2.7.15`: `/usr/bin/python` - `2.7.15`: `/usr/bin/python2` - `3.6.5`: `/usr/bin/python3` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.6.5', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.15.0-23-generic', 'platform_system': 'Linux', 'platform_version': '#25-Ubuntu SMP Wed May 23 18:02:16 UTC 2018', 'python_full_version': '3.6.5', 'python_version': '3.6', 'sys_platform': 'linux'} ``` System environment variables: - `BYOBU_ACCENT` - `BYOBU_BACKEND` - `BYOBU_CHARMAP` - `BYOBU_CONFIG_DIR` - `BYOBU_DARK` - `BYOBU_DATE` - `BYOBU_DISTRO` - `BYOBU_HIGHLIGHT` - `BYOBU_LIGHT` - `BYOBU_PAGER` - `BYOBU_PREFIX` - `BYOBU_PYTHON` - `BYOBU_READLINK` - `BYOBU_RUN_DIR` - `BYOBU_SED` - `BYOBU_TERM` - `BYOBU_TIME` - `BYOBU_TTY` - `BYOBU_ULIMIT` - `BYOBU_WINDOW_NAME` - `CLUTTER_IM_MODULE` - `COLORTERM` - `DBUS_SESSION_BUS_ADDRESS` - `DEFAULTS_PATH` - `DESKTOP_SESSION` - `DISPLAY` - `GDMSESSION` - `GDM_LANG` - `GJS_DEBUG_OUTPUT` - `GJS_DEBUG_TOPICS` - `GNOME_DESKTOP_SESSION_ID` - `GNOME_SHELL_SESSION_MODE` - `GNOME_TERMINAL_SCREEN` - `GNOME_TERMINAL_SERVICE` - `GPG_AGENT_INFO` - `GREP_COLORS` - `GTK_IM_MODULE` - `GTK_MODULES` - `HOME` - `IM_CONFIG_PHASE` - `LANG` - `LANGUAGE` - `LC_ADDRESS` - `LC_IDENTIFICATION` - `LC_MEASUREMENT` - `LC_MONETARY` - `LC_NAME` - `LC_NUMERIC` - `LC_PAPER` - `LC_TELEPHONE` - `LC_TIME` - `LESSCLOSE` - `LESSOPEN` - `LESS_TERMCAP_mb` - `LESS_TERMCAP_md` - `LESS_TERMCAP_me` - `LESS_TERMCAP_se` - `LESS_TERMCAP_so` - `LESS_TERMCAP_ue` - `LESS_TERMCAP_us` - `LOGNAME` - `LS_COLORS` - `MANDATORY_PATH` - `PAPERSIZE` - `PATH` - `PWD` - `QT4_IM_MODULE` - `QT_ACCESSIBILITY` - `QT_IM_MODULE` - `SESSION_MANAGER` - `SHELL` - `SHLVL` - `SSH_AGENT_PID` - `SSH_AUTH_SOCK` - `S_COLORS` - `TERM` - `TEXTDOMAIN` - `TEXTDOMAINDIR` - `TMUX` - `TMUX_PANE` - `USER` - `VTE_VERSION` - `XAUTHORITY` - `XDG_CONFIG_DIRS` - `XDG_CURRENT_DESKTOP` - `XDG_DATA_DIRS` - `XDG_GREETER_DATA_DIR` - `XDG_MENU_PREFIX` - `XDG_RUNTIME_DIR` - `XDG_SEAT` - `XDG_SEAT_PATH` - `XDG_SESSION_DESKTOP` - `XDG_SESSION_ID` - `XDG_SESSION_PATH` - `XDG_SESSION_TYPE` - `XDG_VTNR` - `XMODIFIERS` - `PYTHONDONTWRITEBYTECODE` - `PIP_PYTHON_PATH` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/redacted/go/bin:/home/redacted/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin` - `SHELL`: `/bin/bash` - `LANG`: `en_US.UTF-8` - `PWD`: `/home/redacted/i_am_a_very_nice_project_name_and_this_is_fine` --------------------------- Contents of `Pipfile` ('/home/redacted/i_am_a_very_nice_project_name_and_this_is_fine/Pipfile'): ```toml [[source]] url = "https://pypi.org/simple" verify_ssl = true name = "pypi" [packages] [dev-packages] [requires] python_version = "3.6" ```

[badp]

I should note that, even though I intentionally used a very long directory name in the issue, _subscriptions_99999999_9999_9999_9999_999999999999_resourceGroups_000000_providers_Microsoft_Compute_virtualMachines_test_node_1_InstanceView.json is long enough on its own to cause failures.

~> mkdir x
~> cd x
~/x> pipenv install apache-libcloud
Creating a virtualenv for this project…
Using /usr/bin/python3 (3.6.5) to create virtualenv…
⠋Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/redacted/.local/share/virtualenvs/x-YC8fjoIL/bin/python3
Also creating executable in /home/redacted/.local/share/virtualenvs/x-YC8fjoIL/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/redacted/.local/share/virtualenvs/x-YC8fjoIL
Creating a Pipfile for this project…
Installing apache-libcloud…
Collecting apache-libcloud
  Using cached https://files.pythonhosted.org/packages/56/28/45a9d3bba612fc3f47ef085a14446a4f68f45ce10070562edff1e9291150/apache_libcloud-2.3.0-py2.py3-none-any.whl
Collecting requests (from apache-libcloud)
  Using cached https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests->apache-libcloud)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests->apache-libcloud)
  Using cached https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl
Collecting urllib3<1.24,>=1.21.1 (from requests->apache-libcloud)
  Using cached https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests->apache-libcloud)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Installing collected packages: chardet, certifi, urllib3, idna, requests, apache-libcloud

Error:  An error occurred while installing apache-libcloud!
Could not install packages due to an EnvironmentError: [Errno 36] File name too long: '/home/redacted/.local/share/virtualenvs/x-YC8fjoIL/lib/python3.6/site-packages/libcloud/test/compute/fixtures/azure_arm/_subscriptions_99999999_9999_9999_9999_999999999999_resourceGroups_000000_providers_Microsoft_Compute_virtualMachines_test_node_1_InstanceView.json'

Occasionally (I'm still trying to reproduce this consistently), this fails silently (!!!), causing libcloud to be installed only partially, which causes failures at import time.

[badp]

Oh, this is actually caused by me using ecryptfs, which only supports files up 143 character long. _subscriptions_99999999_9999_9999_9999_999999999999_resourceGroups_000000_providers_Microsoft_Compute_virtualMachines_test_node_1_InstanceView.json is 147 characters.

(╯°□°）╯︵ ┻━┻

[uranusjr]

I was about to say this :) The path limit is usually somewhere in the thousands (2048 and 4096 are common values), so the file name is the problem—and if it is, there is nothing we can do.

You do raise a theoratical issue here though. If the project name is very long (e.g. 140 characters in your case), we can go over the limit by appending the hash. Maybe we should limit the virtualenv name to a more practical length, e.g. 100 characters (totally arbitrary).

[badp]

@uranusjr as a shitty workaround, would it be possible to change the location of my pipenv environment to, say, something under /tmp? pipenvs are supposed to be reproducible, after all, so I wouldn't have too much of a problem having to recreate them on every reboot...

For other unfortunate souls stumbling on this, see also LIBCLOUD-1000

[isotopp]

@uranusjr 8.3 for example :-)

[uranusjr]

@badp Yes, you can set WORKON_HOME=/tmp to change the path prefix.

[grantstephens]

Another work around is to clone https://github.com/apache/libcloud and install it from source as it were.