techalchemy
commented
5 years ago you probably need --site-packages when you pipenv install -- try with the master branch and let us know if this is working for you
Closed tsela closed 2 years ago
techalchemy
commented
5 years ago you probably need --site-packages when you pipenv install -- try with the master branch and let us know if this is working for you
tsela
commented
5 years ago Sorry for not coming back to this earlier. Because of this and other problems I actually moved my development setup to Linux, so I cannot try what you ask. Also, I fail to see what adding --site-packages would do to help, as this issue does not concern system-wide installed packages but rather an incompatibility between Pipenv's tendency to reinstall packages over and over even when unnecessary and Windows's locking DLLs when in use.
IvanaGyro
commented
5 years ago I am working on macOS and I occur the same problem. AFAIK, there are two processes will be run after firing pipenv install [package]. The first is installing and the following is locking(creating Pipfile.lock). In the sconed process, pipenv reinstall all packages listed in Pipfile. This takes a lot of time especially when I have a long list in Pipfile. I don't know the reason for reinstalling all installed packages when creating the lock file, however, I know no developer want to wait for reinstalling all packages just to install only one new package. Below is the version information.
OS: macOS Catalina 10.15 python: 3.7.4 pipenv: master branch at 4ca88493 pip: 19.3.1 virtualenv: 16.7.7
evilUrge
commented
4 years ago have the same issue here, both on osx and alpine
and --site-packages flag doesn't work at all:
bash-4.4# pipenv install --dev --site-packages
Installing --site-packages…
WARNING: Invalid requirement, parse error at "'--site-p'"
✘ Installation Failed
matteius
commented
2 years ago Many moons ago there was this issue. @tsela If you care to check the latest version of pipenv to see if this can be closed?
tsela
commented
2 years ago Sorry, in the meantime I have stopped using Pipenv, so I can't check whether this problem is solved. Maybe the other people on this thread who mentioned having the same problem can chime in, @IvanaGyro and @evilUrge.
Issue description
When installing a new package via Pipenv, after installing that new package Pipenv somehow reinstalls all packages mentioned in the Pipfile (as mentioned by the line "Installing dependencies from Pipfile), even those unrelated to that new package. This may work fine on Unix-based OSes, but on Windows there is that problem that Windows locks access to files that are currently in use by another process, so for instance when trying to install a new package while running a Jupyter notebook, the process will end up failing because the reinstalling step tried to overwrite a locked file, resulting in a "[WinError 5] Access is denied" error.
Note that the new package is usually installed correctly, and normally nothing gets actually broken, but it is annoying, especially since there is no reason why installing a new package should force a reinstall of all packages in the virtual environment, especially the unrelated ones. It makes the process much slower than it could be, and it causes the kind of issues I described above.
Expected result
I have a virtual environment containing a few packages including PyMuPDF (an independent PDF-handling library with no dependencies and no dependents), and a Jupyter notebook running that makes use of PyMuPDF. I want to test another PDF library, PDFQuery, which is independent from PyMuPDF. If I do
pipenv install pdfquery, my expectation is for Pipenv to install it, its dependencies if they are not already present, and I should get a message that the installation has succeeded.Actual result
Instead, here is the result:
As you can see, installing PDFQuery prompted a reinstall of PyMuPDF (despite the two packages being completely unrelated. They don't even have common dependencies), but since PyMuPDF is currently in use in my open Jupyter notebook, the
_fitz.pydfile (a part of PyMuPDF) is locked by Windows, and the attempt to overwrite it causes the[pipenv.exceptions.InstallError]: ["Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'C:\\\\Users\\\\CHRIST~1.GRA\\\\AppData\\\\Local\\\\Temp\\\\pip-uninstall-adfcfhxg\\\\users\\\\christophe.grandsire\\\\.virtualenvs\\\\equipment_mapping-lnlbcdyh\\\\lib\\\\site-packages\\\\fitz\\\\_fitz.pyd'", 'Consider using the--useroption or check the permissions.'], and the followingERROR: ERROR: Package installation failed..., which is particularly egregious since PDFQuery was actually installed correctly and works fine.Steps to replicate
I can't provide a full list of steps to replicate as it would require me to provide the Jupyter notebook I'm using and the data it relies on, and I'm not allowed to share them outside my company. But it should be possible to reproduce this error on any Windows computer (I'm on Windows 10) with a virtual environment containing a specific package currently in use by a Jupyter notebook, and then try to install any other package in that environment. You may need to make sure that specific package contains a
.pydfile, which is basically a Windows DLL and is probably the reason why Windows refuses to overwrite it.Here is the output of
pipenv --support:$ pipenv --support
Pipenv version: `'2018.11.26'` Pipenv location: `'C:\\Users\\Christophe.Grandsire\\AppData\\Roaming\\Python\\Python37\\site-packages\\pipenv'` Python location: `'c:\\program files\\python37\\python.exe'` Python installations found: - `3.7.0`: `C:\Program Files\Python37\python.exe` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.7.0', 'os_name': 'nt', 'platform_machine': 'AMD64', 'platform_python_implementation': 'CPython', 'platform_release': '10', 'platform_system': 'Windows', 'platform_version': '10.0.16299', 'python_full_version': '3.7.0', 'python_version': '3.7', 'sys_platform': 'win32'} ``` System environment variables: - `ABAQUSLM_LICENSE_FILE` - `ACLOCAL_PATH` - `ADSKFLEX_LICENSE_FILE` - `AD_PATH` - `ALLUSERSPROFILE` - `ALT_LICENSE_FILE` - `ANSICON` - `ANSICON_DEF` - `APPDATA` - `BEICIP_LICENSE_FILE` - `BLM_LICENSE_FILE` - `BLUECIEL_LICENSE_FILE` - `BMGEO_LICENSE_FILE` - `CAESAR_LICENSE_FILE` - `CALLMD_LICENSE_FILE` - `CDLMD_LICENSE_FILE` - `CGLD_LICENSE_FILE` - `CMG_LIC_HOST` - `COMMONPROGRAMFILES` - `COMPUTERNAME` - `COMSPEC` - `CONFIG_SITE` - `CUDA_PATH` - `CUDA_PATH_V9_2` - `COMMONPROGRAMFILES(X86)` - `COMMONPROGRAMW6432` - `CONEMUANSI` - `CONEMUANSILOG` - `CONEMUARGS2` - `CONEMUARGS` - `CONEMUBACKHWND` - `CONEMUBASEDIR` - `CONEMUBASEDIRSHORT` - `CONEMUBUILD` - `CONEMUCFGDIR` - `CONEMUCONFIG` - `CONEMUDIR` - `CONEMUDRAWHWND` - `CONEMUDRIVE` - `CONEMUHWND` - `CONEMUHOOKS` - `CONEMUPID` - `CONEMUPALETTE` - `CONEMUSERVERPID` - `CONEMUTASK` - `CONEMUWORKDIR` - `CONEMUWORKDRIVE` - `DHSDELFT_LICENSE_FILE` - `DISPLAY` - `DNVSLM_LICENSE_FILE` - `ELIIS_LICENSE_FILE` - `ENRESINT_LICENSE_FILE` - `EXEPATH` - `FLEXLM_NO_CKOUT_INSTALL_LIC` - `GDAL_DRIVER_PATH` - `GEOCOSM_LICENSE_FILE` - `GEOMATICLM_LICENSE_FILE` - `GEOSOFT_LICENSE_FILE` - `GEOTOMO_LICENSE_FILE` - `HAMP_RUSS_LICENSE_FILE` - `HOME` - `HOMEDRIVE` - `HOMEPATH` - `HOMESHARE` - `HOSTNAME` - `HPQ_LICENSE_FILE` - `HRH_LICENSE_FILE` - `IDL_LMGRD_LICENSE_FILE` - `IHS_LICENSE_FILE` - `IKON_LICENSE_FILE` - `IMSTATUS` - `INFOPATH` - `INTEL_LICENSE_FILE` - `IPASSI_LICENSE_FILE` - `IVS_LICENSE_FILE` - `JASONLM_LICENSE_FILE` - `JAVA_HOME` - `JOA_LICENSE_FILE` - `JUSTINT_LICENSE_FILE` - `KAPPA_LICENSE_FILE` - `KNOWSYS_LICENSE_FILE` - `LANG` - `LCT_LMD_LICENSE_FILE` - `LGCX_LICENSE_FILE` - `LICSRV_LICENSE_FILE` - `LMGRD.SLB_LICENSE_FILE` - `LM_PROJECT` - `LOCALAPPDATA` - `LOGONSERVER` - `LSHOST` - `MANPATH` - `MARINLMD_LICENSE_FILE` - `MCSLMD_LICENSE_FILE` - `MDAXBBWC_LICENSE_FILE` - `MFRACLM_LICENSE_FILE` - `MINGW_CHOST` - `MINGW_PACKAGE_PREFIX` - `MINGW_PREFIX` - `MINITAB_LICENSE_FILE` - `MLM_LICENSE_FILE` - `MSC_LICENSE_FILE` - `MSIPATH1` - `MSIPATH2` - `MSIPATH3` - `MSYSTEM` - `MSYSTEM_CARCH` - `MSYSTEM_CHOST` - `MSYSTEM_PREFIX` - `MVELD_LICENSE_FILE` - `NSITECHN_LICENSE_FILE` - `NUMBER_OF_PROCESSORS` - `NVCUDASAMPLES9_2_ROOT` - `NVCUDASAMPLES_ROOT` - `ODM_LICENSE_FILE` - `OLDPWD` - `OPTASENSE_LICENSE_FILE` - `ORIGINAL_PATH` - `ORIGINAL_TEMP` - `ORIGINAL_TMP` - `OS` - `OSP_LICENSE_FILE` - `ONEDRIVE` - `PALISADE_LICENSE_FILE` - `PATH` - `PATHEXT` - `PETNEWFL_LICENSE_FILE` - `PETROSYS_LICENSE_FILE` - `PGROUPD_LICENSE_FILE` - `PIPENV_SKIP_LOCK` - `PKG_CONFIG_PATH` - `PLINK_PROTOCOL` - `PRDM_GEO_LICENSE_FILE` - `PROCESSOR_ARCHITECTURE` - `PROCESSOR_IDENTIFIER` - `PROCESSOR_LEVEL` - `PROCESSOR_REVISION` - `PROGRAMFILES` - `PS1` - `PSELMD_LICENSE_FILE` - `PSG_LICENSE_FILE` - `PSMODULEPATH` - `PTC_D_LICENSE_FILE` - `PUBLIC` - `PVTLMD_LICENSE_FILE` - `PWD` - `PROGRAMDATA` - `PROGRAMFILES(X86)` - `PROGRAMW6432` - `QEDILM_LICENSE_FILE` - `REALCHRT_LICENSE_FILE` - `RECALL5_LICENSE_FILE` - `RLM_LICENSE` - `ROPTIMA_LICENSE_FILE` - `SAFE_LICENSE_FILE` - `SBT_HOME` - `SCPLMD_LICENSE_FILE` - `SESSIONNAME` - `SHELL` - `SHLVL` - `SIEPRTS_LICENSE_FILE` - `SLBFD_LICENSE_FILE` - `SLBSLS_LICENSE_FILE` - `SNC_LIB` - `SNC_LIB_64` - `SPLMCS_LICENSE_FILE` - `SSF_LIBRARY_PATH` - `SSF_LIBRARY_PATH_64` - `SSH_ASKPASS` - `STACK_ROOT` - `STA_FLXM_LICENSE_FILE` - `STRATADATA_LICENSE` - `STRSOLV_LICENSE_FILE` - `SWRAP_LICENSE_FILE` - `SYSTEMDRIVE` - `SYSTEMROOT` - `TEMP` - `TERM` - `TESSDATA_PREFIX` - `THERMOCORP_LICENSE_FILE` - `TMP` - `TMPDIR` - `TNSNAMES` - `TNS_ADMIN` - `TOKEN_LICENSE_FILE` - `TRAMONTA_LICENSE_FILE` - `UATDATA` - `USERDNSDOMAIN` - `USERDOMAIN` - `USERDOMAIN_ROAMINGPROFILE` - `USERNAME` - `USERPROFILE` - `VRCONTXT_LICENSE_FILE` - `WINDIR` - `_` - `HTTP_PROXY` - `HTTPS_PROXY` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv�specific environment variables: - `PIPENV_SKIP_LOCK`: `True` Debug�specific environment variables: - `PATH`: `C:\Users\Christophe.Grandsire\bin;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\local\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Git\cmd;C:\Program Files\Git\usr\bin;C:\Program Files\ConEmu\ConEmu\Scripts;C:\Program Files\ConEmu;C:\Program Files\ConEmu\ConEmu;C:\ProgramData\DockerDesktop\version-bin;C:\Program Files\Docker\Docker\Resources\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp;C:\Program Files\Python37\Scripts;C:\Program Files\Python37;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0;C:\Program Files\Intel\WiFi\bin;C:\Program Files\Common Files\Intel\WirelessCommon;C:\Program Files\1E\NomadBranch;C:\Program Files (x86)\Plantronics\Spokes3G;C:\Program Files\MATLAB\R2018a\runtime\win64;C:\Program Files\MATLAB\R2018a\bin;C:\Program Files\dotnet;C:\Program Files\PuTTY;C:\Program Files\Java\jdk-11.0.1\bin;C:\Program Files (x86)\sbt\bin;C:\Program Files\Tesseract-OCR;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Users\Christophe.Grandsire\AppData\Roaming\Python\Python37\Scripts;C:\Users\Christophe.Grandsire\AppData\Roaming\local\bin;C:\Users\Christophe.Grandsire\AppData\Local\Microsoft\WindowsApps;C:\Users\Christophe.Grandsire\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Christophe.Grandsire\AppData\Local\GitHubDesktop\bin;C:\Users\Christophe.Grandsire\.virtualenvs\Global_Isopach-OUJ2C1wF\Lib\site-packages\osgeo;C:\Users\Christophe.Grandsire\Julia-1.0.2\bin;C:\Program Files\Git\usr\bin\vendor_perl;C:\Program Files\Git\usr\bin\core_perl` - `SHELL`: `C:\Program Files\Git\usr\bin\bash.exe` - `LANG`: `en_US.UTF-8` - `PWD`: `C:/Users/Christophe.Grandsire/Cases/Code/Equipment Mapping` --------------------------- Contents of `Pipfile` ('C:\\Users\\Christophe.Grandsire\\Cases\\Code\\Equipment Mapping\\Pipfile'): ```toml [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev-packages] pylint = "*" pydocstyle = "*" rope = "*" jupyter = "*" [packages] pymupdf = "*" pdfplumber = "*" regex = "*" numpy = {path = "C:/Users/Christophe.Grandsire/Downloads/numpy-1.15.4-cp37-cp37m-win_amd64.whl"} pandas = "*" xlrd = "*" pdfquery = "*" [requires] python_version = "3.7" ```Thanks in advance for your help!