Closed skyoo2003 closed 2 years ago
techalchemy
commented
4 years ago Can you verify that you actually can git clone https://${GITHUB_TOKEN}@github.com/skyoo2003/myprivate.git using the same environment variable? I have just now confirmed that environment variables do get translated from requirements.txt files.
skyoo2003
commented
4 years ago @techalchemy
After setting the environment variable GITHUB_TOKEN, git clone command worked successfully.
$ export GITHUB_TOKEN="{My Github personal access token}"
$ LC_ALL=en_us.UTF-8 git clone "https://${GITHUB_TOKEN}@github.com/skyoo2003/myprivate.git"
Cloning into 'myprivate'...
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 7 (delta 0), reused 7 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), done.However, after setting the environment variable, I retried the command "pipenv install -r requirements.txt --verbose" and failed for the same reason.
$ export GITHUB_TOKEN="{My Github personal access token}"
$ pipenv install -r requirements.txt --verbose
Requirements file provided! Importing into Pipfile…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
⠴ Locking...
⠧ Locking...
⠙ Locking...
⠸ Locking...ERROR:pip.subprocessor:Command errored out with exit status 128:
command: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-44kpmqn3-src/myprivate
cwd: None
Complete output (2 lines):
remote: Invalid username or password.
fatal: Authentication failed for 'https://****@github.com/skyoo2003/myprivate.git/'
----------------------------------------
✘ Locking Failed!
Traceback (most recent call last):
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 807, in <module>
main()
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 803, in main
parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 785, in _main
resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 753, in resolve_packages
requirements_dir=requirements_dir,
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 736, in resolve
req_dir=requirements_dir
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1379, in resolve_deps
req_dir=req_dir,
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1084, in actually_resolve_deps
deps, index_lookup, markers_lookup, project, sources, req_dir, clear, pre
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 649, in create
pre=pre, clear=clear
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 463, in get_metadata
req, resolver=transient_resolver
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 540, in get_deps_from_req
req_list, lockfile = get_vcs_deps(reqs=[req])
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1867, in get_vcs_deps
with temp_path(), locked_repository(requirement) as repo:
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 2029, in locked_repository
with requirement.req.locked_vcs_repo(src_dir=src_dir) as repo:
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2141, in locked_vcs_repo
vcsrepo = self.get_vcs_repo(src_dir=src_dir)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2097, in get_vcs_repo
vcsrepo.obtain()
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/vcs.py", line 77, in obtain
self.repo_backend.obtain(self.checkout_directory, self.parsed_url)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 527, in obtain
self.fetch_new(dest, url, rev_options)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/git.py", line 225, in fetch_new
self.run_command(make_command('clone', '-q', url, dest))
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 667, in run_command
log_failed_cmd=log_failed_cmd)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/utils/subprocess.py", line 242, in call_subprocess
raise InstallationError(exc_msg)
pipenv.patched.notpip._internal.exceptions.InstallationError: Command errored out with exit status 128: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-44kpmqn3-src/myprivate Check the logs for full command output.
ERROR:pip.subprocessor:Command errored out with exit status 128:
command: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-44kpmqn3-src/myprivate
cwd: None
Complete output (2 lines):
remote: Invalid username or password.
fatal: Authentication failed for 'https://****@github.com/skyoo2003/myprivate.git/'
----------------------------------------
Traceback (most recent call last):
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 807, in <module>
main()
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 803, in main
parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 785, in _main
resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 753, in resolve_packages
requirements_dir=requirements_dir,
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/resolver.py", line 736, in resolve
req_dir=requirements_dir
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1379, in resolve_deps
req_dir=req_dir,
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1084, in actually_resolve_deps
deps, index_lookup, markers_lookup, project, sources, req_dir, clear, pre
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 649, in create
pre=pre, clear=clear
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 463, in get_metadata
req, resolver=transient_resolver
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 540, in get_deps_from_req
req_list, lockfile = get_vcs_deps(reqs=[req])
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 1867, in get_vcs_deps
with temp_path(), locked_repository(requirement) as repo:
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/utils.py", line 2029, in locked_repository
with requirement.req.locked_vcs_repo(src_dir=src_dir) as repo:
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2141, in locked_vcs_repo
vcsrepo = self.get_vcs_repo(src_dir=src_dir)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2097, in get_vcs_repo
vcsrepo.obtain()
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/vcs.py", line 77, in obtain
self.repo_backend.obtain(self.checkout_directory, self.parsed_url)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 527, in obtain
self.fetch_new(dest, url, rev_options)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/git.py", line 225, in fetch_new
self.run_command(make_command('clone', '-q', url, dest))
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/vcs/versioncontrol.py", line 667, in run_command
log_failed_cmd=log_failed_cmd)
File "/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv/patched/notpip/_internal/utils/subprocess.py", line 242, in call_subprocess
raise InstallationError(exc_msg)
pipenv.patched.notpip._internal.exceptions.InstallationError: Command errored out with exit status 128: git clone -q 'https://****@github.com/skyoo2003/myprivate.git' /tmp/pipenv-44kpmqn3-src/myprivate Check the logs for full command output.
LanceSandino
commented
4 years ago I'm having the same issue when using git+https://raw_token@github.com/user/package-name.git@branch#egg=package-name
This is from AWS Beanstalk Python 3.7, but also fails on my mac
Requirements file provided! Importing into Pipfile…
An error occurred while installing -e git+https://****@github.com/user/package.git@branch#egg=package! Will try again.
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 252, in install
[InstallError]: site_packages=state.site_packages
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 2062, in do_install
[InstallError]: keep_outdated=keep_outdated
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1314, in do_init
[InstallError]: pypi_mirror=pypi_mirror,
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 901, in do_install_dependencies
[InstallError]: retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Obtaining package from git+https://****@github.com/user/package.git@branch#egg=package (from -r /tmp/pipenv-k0wkmnw2-requirements/pipenv-cpaca5jp-requirement.txt (line 1))
[pipenv.exceptions.InstallError]: Cloning https://****@github.com/user/package.git@branch#egg=package (to revision process-bookings) to /var/app/venv/staging-LQM1lest/src/package
[pipenv.exceptions.InstallError]: Running command git clone -q 'https://****@github.com/user/package.git@branch#egg=package' /var/app/venv/staging-LQM1lest/src/package
[pipenv.exceptions.InstallError]: fatal: could not read Password for 'https://****@github.com': No such device or address
[pipenv.exceptions.InstallError]: ERROR: Command errored out with exit status 128: git clone -q 'https://****@github.com/user/package.git@branch#egg=package' /var/app/venv/staging-LQM1lest/src/packageCheck the logs for full command output.
ERROR: Couldn't install package: package
Package installation failed...
2020/07/17 02:22:27.241083 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c python3 -m pipenv install -r requirements.txt --skip-lock failed with error exit status 1. Stderr:Creating a Pipfile for this project…
Requirements file provided! Importing into Pipfile…
An error occurred while installing -e git+https://****@github.com/user/package.git@branch#egg=package! Will try again.
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 252, in install
[InstallError]: site_packages=state.site_packages
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 2062, in do_install
[InstallError]: keep_outdated=keep_outdated
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1314, in do_init
[InstallError]: pypi_mirror=pypi_mirror,
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 901, in do_install_dependencies
[InstallError]: retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Obtaining package from git+https://****@github.com/user/package.git@branch#egg=package (from -r /tmp/pipenv-k0wkmnw2-requirements/pipenv-cpaca5jp-requirement.txt (line 1))
[pipenv.exceptions.InstallError]: Cloning https://****@github.com/user/package.git@branch#egg=package (to revision process-bookings) to /var/app/venv/staging-LQM1lest/src/package
[pipenv.exceptions.InstallError]: Running command git clone -q 'https://****@github.com/user/package.git@branch#egg=package' /var/app/venv/staging-LQM1lest/src/package
[pipenv.exceptions.InstallError]: fatal: could not read Password for 'https://****@github.com': No such device or address
[pipenv.exceptions.InstallError]: ERROR: Command errored out with exit status 128: git clone -q 'https://****@github.com/user/package.git@branch#egg=package' /var/app/venv/staging-LQM1lest/src/package
Check the logs for full command output.
ERROR: Couldn't install package: package
Package installation failed...Works fine with pip, guess it's related to pipenv for some reason.... (first time using pipenv)
CaselIT
commented
4 years ago It also does not work when using basic auth to authenticate to a git provider.
The problem seems to be that the command that is run has the sensible parts (token/password) sanitized.
Example is a line in the requirement file with a repository and an user and password in base auth http form, like
https://gitlab+deploy-token-123:foobar@gitlab.com/some/project.git#egg=some-eggThe command that is run by subprocess is
['git', 'clone', '-q', 'https://gitlab+deploy-token-123:****@gitlab.com/some/project.git', '/tmp/pipenv-asf23-src/project']that clearly does not work That command line is from this file https://github.com/pypa/pipenv/blob/b29a4884e24dd2dd72f379d64b23779a676d2ef9/pipenv/patched/notpip/_internal/utils/subprocess.py#L186-L191
CaselIT
commented
4 years ago The problem in the example with basic auth seems to be that the args in cmd are sanitized at least twice and HiddenText.secret contains an already sanitized string
Edit: version 2018.11.26 seems to work
LanceSandino
commented
4 years ago any update on this?
sandeep-tukaram
commented
3 years ago Getting same error with $ pipenv --version pipenv, version 2020.11.15
Executing git clone with user cred works. Executing pipenv install with env vairable for credentials also doesn't work
matteius
commented
2 years ago Can this be checked with the new pipenv requirements command which replaced the pipenv lock -r? Version pipenv==2022.8.19
Issue description
I just tried to import the following the file, but failed.
Expected result
Expect the dependency to be included in "Pipfile" and successful installation
Actual result
I typed the enter key twice because it didn't seem to stop, and it failed with the error message below.
Steps to replicate
Provide the steps to replicate (which usually at least includes the commands and the Pipfile).
pipenv install -r requirements.txt --verbose$ pipenv --support
Pipenv version: `'2020.4.1b1'` Pipenv location: `'/home/lukas/.pyenv/versions/3.6.10/lib/python3.6/site-packages/pipenv'` Python location: `'/home/lukas/.pyenv/versions/3.6.10/bin/python3.6'` Python installations found: - `3.6.10`: `/home/lukas/.pyenv/versions/3.6.10/bin/python3.6` - `3.6.10`: `/home/lukas/.pyenv/versions/3.6.10/bin/python3.6m` - `3.6.10`: `/home/lukas/.pyenv/versions/3.6.10/bin/python3` - `3.6.10`: `/home/lukas/.pyenv/versions/3.6.10/bin/python3.6` - `3.6.9`: `/usr/bin/python3.6` - `3.6.9`: `/usr/bin/python3.6m` - `3.6.9`: `/usr/bin/python3` - `2.7.17`: `/usr/bin/python2` - `2.7.17`: `/usr/bin/python2.7` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.6.10', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '5.3.0-51-generic', 'platform_system': 'Linux', 'platform_version': '#44~18.04.2-Ubuntu SMP Thu Apr 23 14:27:18 UTC 2020', 'python_full_version': '3.6.10', 'python_version': '3.6', 'sys_platform': 'linux'} ``` System environment variables: - `CLUTTER_IM_MODULE` - `LS_COLORS` - `LC_MEASUREMENT` - `LC_PAPER` - `LC_MONETARY` - `XDG_MENU_PREFIX` - `LANG` - `LESS` - `DISPLAY` - `PYENV_ROOT` - `OLDPWD` - `GNOME_SHELL_SESSION_MODE` - `MAVEN_HOME` - `COLORTERM` - `DESKTOP_AUTOSTART_ID` - `PYENV_VIRTUALENV_INIT` - `USERNAME` - `JAVA_HOME` - `PYENV_HOOK_PATH` - `XDG_VTNR` - `ZSH` - `SSH_AUTH_SOCK` - `MANDATORY_PATH` - `LC_NAME` - `XDG_SESSION_ID` - `USER` - `PYENV_DIR` - `PAGER` - `LSCOLORS` - `DESKTOP_SESSION` - `QT4_IM_MODULE` - `TEXTDOMAINDIR` - `GNOME_TERMINAL_SCREEN` - `DEFAULTS_PATH` - `PWD` - `HOME` - `TEXTDOMAIN` - `SSH_AGENT_PID` - `PYENV_VERSION` - `QT_ACCESSIBILITY` - `XDG_SESSION_TYPE` - `XDG_DATA_DIRS` - `XDG_SESSION_DESKTOP` - `LC_ADDRESS` - `LC_NUMERIC` - `GTK_MODULES` - `PAPERSIZE` - `WINDOWPATH` - `VTE_VERSION` - `TERM` - `SHELL` - `QT_IM_MODULE` - `XMODIFIERS` - `IM_CONFIG_PHASE` - `XDG_CURRENT_DESKTOP` - `GPG_AGENT_INFO` - `GNOME_TERMINAL_SERVICE` - `PYENV_SHELL` - `XDG_SEAT` - `SHLVL` - `LANGUAGE` - `LC_TELEPHONE` - `GDMSESSION` - `GNOME_DESKTOP_SESSION_ID` - `LOGNAME` - `DBUS_SESSION_BUS_ADDRESS` - `XDG_RUNTIME_DIR` - `XAUTHORITY` - `XDG_CONFIG_DIRS` - `PATH` - `LC_IDENTIFICATION` - `SESSION_MANAGER` - `GTK_IM_MODULE` - `LC_TIME` - `PIP_DISABLE_PIP_VERSION_CHECK` - `PYTHONDONTWRITEBYTECODE` - `PIP_SHIMS_BASE_MODULE` - `PIP_PYTHON_PATH` - `PYTHONFINDER_IGNORE_UNSUPPORTED` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/lukas/.pyenv/versions/3.6.10/bin:/home/lukas/.pyenv/libexec:/home/lukas/.pyenv/plugins/python-build/bin:/home/lukas/.pyenv/plugins/pyenv-virtualenv/bin:/home/lukas/.pyenv/plugins/pyenv-update/bin:/home/lukas/.pyenv/plugins/pyenv-installer/bin:/home/lukas/.pyenv/plugins/pyenv-doctor/bin:/usr/lib/jvm/adoptopenjdk-8-openj9-amd64/bin:/usr/lib/maven/apache-maven-3.6.3/bin:/home/lukas/.pyenv/plugins/pyenv-virtualenv/shims:/home/lukas/.pyenv/shims:/home/lukas/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/lukas/.fzf/bin` - `SHELL`: `/usr/bin/zsh` - `LANG`: `ko_KR.UTF-8` - `PWD`: `/home/lukas/pipenv-test` --------------------------- Contents of `Pipfile` ('/home/lukas/pipenv-test/Pipfile'): ```toml [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev-packages] [packages] myprivate = {editable = true, git = "https://****@github.com/skyoo2003/myprivate.git", ref = "v1.0.0"} [requires] python_version = "3.6" ```