search

pypa / pipenv

Python Development Workflow for Humans.
https://pipenv.pypa.io
MIT License
24.87k stars 1.87k forks source link

Installation of private repo fails with error: neither 'setup.py' nor 'pyproject.toml' found #6045

Closed DanielSwain closed 10 months ago

DanielSwain commented 10 months ago

Issue description

I know that a problem with installing private repos cropped up in late summer 2023, and it had been fixed for a while. For two or three months I was able to install a private repo with: python3 -m pipenv install git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master

I recently updated pipenv and went back to working on our library (I didn't notice the previous version of pipenv during the update), and the above install line stopped working.

Expected result

The above install had been working to cause the private repo to be installed correctly, and it resulted in an entry in Pipfile like this: our-private-repo = {ref = "master", git = "ssh://git@bitbucket.org/our_organization/our-private-repo.git"}

Actual result

In order to try to get things to work, I then added egg= to the install command: python3 -m pipenv install git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master#egg=our-private-repo --verbose and went back through all Pipenv versions with the above install. It finally worked at version 2023.7.23.

I did find this issue where @matteius mentioned that the egg syntax has been deprecated, so I then tried: python3 -m pipenv install git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master#subdirectory=our-private-repo --verbose with a more recent version of pipenv. That resulted in the following traceback which includes a message that says no setup.py or pyproject.toml file could be found. However, both files are in the repo.

NOTE: I am also using wagtail-django-recaptcha = {ref = "master", git = "git+https://github.com/torchbox-forks/wagtail-django-recaptcha"} in this project that is using our-private-repo.

As an aside, it would be nice if the installation of private repos would be explained in the docs (including information about egg and subdirectory).

INFO:pipenv.patched.pip._internal.vcs.git:Resolved ssh://****@bitbucket.org/our_organization/our-private-repo.git to commit cef1de6c3t8juct393tc9ut4c3c9ut43165cbd
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pipenv/utils/resolver.py", line 440, in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 513, in collect_root_requirements
    reqs = list(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 474, in _make_requirements_from_install_req
    cand = self._make_candidate_from_link(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 211, in _make_candidate_from_link
    self._link_candidate_cache = LinkCandidate(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
    super().__init__(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 157, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 226, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/distributions/sdist.py", line 37, in prepare_distribution_metadata
    self.req.load_pyproject_toml()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/req/req_install.py", line 506, in load_pyproject_toml
    pyproject_toml_data = load_pyproject_toml(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/pyproject.py", line 57, in load_pyproject_toml
    raise InstallationError(
pipenv.patched.pip._internal.exceptions.InstallationError: our-private-repo@ git+ssh://****@bitbucket.org/our_organization/our-private-repo.git@master#subdirectory=our-private-repo
from git+ssh://****@bitbucket.org/our_organization/our-private-repo.git@master#subdirectory=our-private-repo (from -r
/tmp/pipenv-g9xishwo-requirements/pipenv-7s70sjp9-constraints.txt (line 11)) does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

The following output is the result of using the egg syntax as part of the install line:

user@MY-DESKTOP:/path/to/my_site# python3 -m pipenv install git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master#egg=our-private-repo --verbose
Loading .env environment variables...
Installing git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master#egg=our-private-repo...
Resolving git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master#egg=our-private-repo...
Added our-private-repo to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock (bb9326) out of date, updating to (dc0032)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
INFO:pipenv.patched.pip._internal.vcs.git:Cloning ssh://****@bitbucket.org/our_organization/our-private-repo.git (to revision master) to /tmp/tmpnoymflc1
INFO:pip.subprocessor:Running command git clone --filter=blob:none 'ssh://****@bitbucket.org/our_organization/our-private-repo.git' /tmp/tmpnoymflc1
INFO:pip.subprocessor:Cloning into '/tmp/tmpnoymflc1'...
INFO:pip.subprocessor:warning: filtering not recognized by server, ignoring
INFO:pipenv.patched.pip._internal.vcs.git:Resolved ssh://****@bitbucket.org/our_organization/our-private-repo.git to commit cef1de658c7257512babc1e337446e3f6c165cbd
INFO:pipenv.patched.pip._internal.vcs.git:Cloning https://github.com/torchbox-forks/wagtail-django-recaptcha (to revision master) to /tmp/tmpba1byqra
INFO:pip.subprocessor:Running command git clone --filter=blob:none https://github.com/torchbox-forks/wagtail-django-recaptcha /tmp/tmpba1byqra
INFO:pip.subprocessor:Cloning into '/tmp/tmpba1byqra'...
INFO:pipenv.patched.pip._internal.vcs.git:Resolved https://github.com/torchbox-forks/wagtail-django-recaptcha to commit c23696f5c55824a8abd86e13f13da4a90d437d75
INFO:pipenv.patched.pip._internal.operations.prepare:Collecting our-private-repo@ git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master (from -r
/tmp/pipenv-k1xb72tc-requirements/pipenv-hl395gwe-constraints.txt (line 2))
INFO:pipenv.patched.pip._internal.vcs.git:Cloning ssh://****@bitbucket.org/our_organization/our-private-repo.git (to revision master) to
/tmp/pip-temp-y0fpm097/our-private-repo_4a49723870f849b392ae737a26c32337
INFO:pip.subprocessor:Running command git clone --filter=blob:none --quiet 'ssh://****@bitbucket.org/our_organization/our-private-repo.git'
/tmp/pip-temp-y0fpm097/our-private-repo_4a49723870f849b392ae737a26c32337
INFO:pip.subprocessor:warning: filtering not recognized by server, ignoring
INFO:pipenv.patched.pip._internal.vcs.git:Resolved ssh://****@bitbucket.org/our_organization/our-private-repo.git to commit cef1de658c7257512babc1e337446e3f6c165cbd
INFO:pipenv.patched.pip._internal.cli.spinners:Installing build dependencies: started
INFO:pipenv.patched.pip._internal.cli.spinners:Installing build dependencies: finished with status 'done'
INFO:pipenv.patched.pip._internal.cli.spinners:Getting requirements to build wheel: started
INFO:pipenv.patched.pip._internal.cli.spinners:Getting requirements to build wheel: finished with status 'error'
ERROR:pip.subprocessor: Getting requirements to build wheel exited with 1
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pipenv/utils/resolver.py", line 439, in resolve
    results = resolver.resolve(self.constraints, check_supported_wheels=False)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
    req = self._make_requirement_from_install_req(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 453, in _make_requirement_from_install_req
    cand = self._make_candidate_from_link(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache = LinkCandidate(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
    super().__init__(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 157, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 226, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 538, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 653, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/distributions/sdist.py", line 48, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/distributions/sdist.py", line 118, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/distributions/sdist.py", line 95, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/utils/misc.py", line 697, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_vendor/pyproject_hooks/_impl.py", line 311, in _call_hook
    self._subprocess_runner(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/utils/subprocess.py", line 252, in runner
    call_subprocess(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/patched/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pipenv.patched.pip._internal.exceptions.InstallationSubprocessError: Getting requirements to build wheel exited with 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pipenv/resolver.py", line 668, in <module>
    main()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/resolver.py", line 654, in main
    _main(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/resolver.py", line 641, in _main
    resolve_packages(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/resolver.py", line 608, in resolve_packages
    results, resolver = resolve(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/resolver.py", line 588, in resolve
    return resolve_deps(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/utils/resolver.py", line 841, in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/utils/resolver.py", line 614, in actually_resolve_deps
    resolver.resolve()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/utils/resolver.py", line 441, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: [31m[1mERROR[0m: [33mGetting requirements to build wheel exited with 1[0m
✘ Locking Failed!
⠏ Locking...
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/__main__.py", line 4, in <module>
    cli()
  File "/usr/local/lib/python3.8/dist-packages/pipenv/vendor/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/vendor/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/vendor/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/dist-packages/pipenv/vendor/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/cli/command.py", line 209, in install
    do_install(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/routines/install.py", line 292, in do_install
    raise e
  File "/usr/local/lib/python3.8/dist-packages/pipenv/routines/install.py", line 277, in do_install
    do_init(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/routines/install.py", line 614, in do_init
    do_lock(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/routines/lock.py", line 65, in do_lock
    venv_resolve_deps(
  File "/usr/local/lib/python3.8/dist-packages/pipenv/utils/resolver.py", line 782, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/usr/local/lib/python3.8/dist-packages/pipenv/utils/resolver.py", line 651, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

Pipenv location: '/usr/local/lib/python3.8/dist-packages/pipenv'

Python location: '/usr/bin/python3'

OS Name: 'posix'

User pip version: '23.3.1'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.10',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.10.102.1-microsoft-standard-WSL2',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Wed Mar 2 00:30:59 UTC 2022',
 'python_full_version': '3.8.10',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

Pipenv–specific environment variables:

Debug–specific environment variables:

Contents of Pipfile ('/our_info/Pipfile'):

[[source]]
name = "pypi"
verify_ssl = true
url = "https://pypi.python.org/simple"

[packages]
django-allauth = "*"
django-anymail = {extras = ["mailgun"]}
django = {extras = ["argon2"]}
wagtail = "*"
psycopg2-binary = "*"
django-environ = "*"
whitenoise = "*"
wagtail-modeladmin = "*"
django-allauth-2fa = "*"
wagtail-django-recaptcha = {ref = "master", git = "git+https://github.com/torchbox-forks/wagtail-django-recaptcha"}
our-private-repo = {ref = "master", git = "ssh://git@bitbucket.org/our_organization/our-private-repo.git"}

[dev-packages]

[requires]
python_version = "3.8"

Contents of Pipfile.lock ('/path/to/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "69b863f1d077b2c0c28177c9ebcbffc2436167e9596131de1945815ff0bb9326"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
......
......
    },
    "develop": {}
}

DanielSwain commented 10 months ago

I'm glad to be able to close this issue. If anyone wants to use setuptools to build a library, I suggest ignoring the Django instructions (which call for creating setup.cfg and setup.py files in addition to pyproject.toml). Just follow the setuptools instructions here. You only need a pyproject.toml file. When I rebuilt the library using python3 -m build --sdist following the above instructions, I was then able to install it using the latest pipenv (2023.11.15) with the command that had formerly been working: python3 -m pipenv install git+ssh://git@bitbucket.org/our_organization/our-private-repo.git@master.