Repository "private" does not exist | Since 1.1.0

RobinFrcd commented 4 years ago

[x] I am on the latest Poetry version: tested from 1.1.0 to 1.1.2
[x] I have searched the issues of this repo and believe that this is not a duplicate.
- https://github.com/python-poetry/poetry/issues/2109 - closed
- https://github.com/python-poetry/poetry/issues/2110 - closed
[x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
OS version and name: Docker, python:3.7-buster
Poetry version: 1.1.0 to 1.1.2

Simple pyproject.toml with a private repo:

[[tool.poetry.source]]
name = "private"
url = "https://my.private.repo/simple"
secondary = true

Issue

When running a poetry install with a private repo (poetry config http-basic.private is set correctly), I get:

  Stack trace:

  7  /usr/local/lib/python3.7/site-packages/poetry/installation/executor.py:199 in _execute_operation
      197│
      198│             try:
    → 199│                 result = self._do_execute_operation(operation)
      200│             except EnvCommandError as e:
      201│                 if e.e.returncode == -2:

  6  /usr/local/lib/python3.7/site-packages/poetry/installation/executor.py:273 in _do_execute_operation
      271│             return 0
      272│
    → 273│         result = getattr(self, "_execute_{}".format(method))(operation)
      274│
      275│         if result != 0:

  5  /usr/local/lib/python3.7/site-packages/poetry/installation/executor.py:408 in _execute_install
      406│
      407│     def _execute_install(self, operation):  # type: (Install) -> None
    → 408│         return self._install(operation)
      409│
      410│     def _execute_update(self, operation):  # type: (Update) -> None

  4  /usr/local/lib/python3.7/site-packages/poetry/installation/executor.py:434 in _install
      432│             archive = self._download_link(operation, Link(package.source_url))
      433│         else:
    → 434│             archive = self._download(operation)
      435│
      436│         operation_message = self.get_operation_message(operation)

  3  /usr/local/lib/python3.7/site-packages/poetry/installation/executor.py:575 in _download
      573│
      574│     def _download(self, operation):  # type: (Operation) -> Path
    → 575│         link = self._chooser.choose_for(operation.package)
      576│
      577│         return self._download_link(operation, link)

  2  /usr/local/lib/python3.7/site-packages/poetry/installation/chooser.py:60 in choose_for
       58│         """
       59│         links = []
    →  60│         for link in self._get_links(package):
       61│             if link.is_wheel and not Wheel(link.filename).is_supported_by_environment(
       62│                 self._env

  1  /usr/local/lib/python3.7/site-packages/poetry/installation/chooser.py:92 in _get_links
       90│                 repository = self._pool.repository("pypi")
       91│         else:
    →  92│             repository = self._pool.repository(package.source_reference)
       93│
       94│         links = repository.find_links_for_package(package)

  ValueError

  Repository "private" does not exist.

  at /usr/local/lib/python3.7/site-packages/poetry/repositories/pool.py:53 in repository
       49│
       50│         if name in self._lookup:
       51│             return self._repositories[self._lookup[name]]
       52│
    →  53│         raise ValueError('Repository "{}" does not exist.'.format(name))
       54│
       55│     def add_repository(
       56│         self, repository, default=False, secondary=False
       57│     ):  # type: (Repository, bool, bool) -> Pool

If I downgrade to 1.0.10, everything works fine. This error popped at the 1.1.0

abn commented 4 years ago

@RobinFrcd can you provide a sanitised output of poetry config repository please?

RobinFrcd commented 4 years ago

{'private': {'url': 'https://my.private.repo/simple/'}}

abn commented 4 years ago

@RobinFrcd I am unable to reproduce this.

root@610e2455d140:/foobar# poetry add --source private foobar^C
root@610e2455d140:/foobar# cat pyproject.toml | tail -n 10

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[[tool.poetry.source]]
name = "private"
url = "https://my.private.repo/simple"
secondary = true

root@610e2455d140:/foobar# poetry config repositories
{'private': {'url': 'https://my.private.repo/simple'}}
root@610e2455d140:/foobar# poetry  config http-basic.private username password
No suitable keyring backends were found
Using a plaintext file to store and retrieve credentials
root@610e2455d140:/foobar# poetry add --source private foobar

  ConnectionError

  HTTPSConnectionPool(host='my.private.repo', port=443): Max retries exceeded with url: /simple/foobar/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f84fe257040>: Failed to establish a new connection: [Errno -2] Name or service not known'))

  at /usr/local/lib/python3.8/site-packages/requests/adapters.py:516 in send
      512│             if isinstance(e.reason, _SSLError):
      513│                 # This branch is for urllib3 v1.22 and later.
      514│                 raise SSLError(e, request=request)
      515│ 
    → 516│             raise ConnectionError(e, request=request)
      517│ 
      518│         except ClosedPoolError as e:
      519│             raise ConnectionError(e, request=request)
      520│

RobinFrcd commented 4 years ago

Well, the URL I gave as an example is not a valid one. I can't give the private repo I'm using for privacy reasons. The error I get is on the poetry install, not the add --source.

abn commented 4 years ago

@RobinFrcd yeah; I am aware; i just want to demonstrate that the code got past the credential fetching.

abn commented 4 years ago

Can you maybe try in a container or a clean environent? Just want to make sure this is not somehting environment specific.

RobinFrcd commented 4 years ago

Alright, I managed to make it work by tweaking the toml. I moved tool.poetry.dependencies to the top.

I had (which is working on 1.0.10 but not on 1.1.0+)

[[tool.poetry.source]]
name = "private"
url = "https://my.private.repo/simple/"
secondary = true

[tool.poetry]
name = "metrics"
version = "0"
description = ""
authors = [""]

[tool.black]
line-length = 88
target-version = ['py37']
include = '\.pyi?$'
exclude = '''
/(
    \.eggs
  | \.git
  | \.hg
  | \.mypy_cache
  | \.tox
  | \.venv
  | _build
  | buck-out
  | build
  | dist
)/
'''

[tool.coverage.run]
omit = [
    'tests/*',
    '*_pb2.py',
    '*_pb2_grpc.py'
]

[tool.poetry.dependencies]
python = "^3.7"
google-api-python-client = "==1.9.1"
grpcio = "==1.29.0"
grpcio-status = "==1.29.0"
private-logger = "==2.0.2"
protobuf = "==3.11.1"
sentry-sdk = "==0.15.1"
taskipy = ">=1.3.0"
numpy = "*"

[tool.poetry.dev-dependencies]
black = "==19.3b0"
flake8 = "==3.7.9"
mypy = "==0.770"

[tool.taskipy.tasks]
black = "black ."
mypy = "mypy . --ignore-missing-imports"
linter = "flake8 ."

And switched it to

[[tool.poetry.source]]
name = "private"
url = "https://my.private.repo/simple/"
secondary = true

[tool.poetry]
name = "metrics"
version = "0"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = "^3.7"
google-api-python-client = "==1.9.1"
grpcio = "==1.29.0"
grpcio-status = "==1.29.0"
private-logger = "==2.0.2"
protobuf = "==3.11.1"
sentry-sdk = "==0.15.1"
taskipy = ">=1.3.0"
numpy = "*"

[tool.poetry.dev-dependencies]
black = "==19.3b0"
flake8 = "==3.7.9"
mypy = "==0.770"

[tool.black]
line-length = 88
target-version = ['py37']
include = '\.pyi?$'
exclude = '''
/(
    \.eggs
  | \.git
  | \.hg
  | \.mypy_cache
  | \.tox
  | \.venv
  | _build
  | buck-out
  | build
  | dist
)/
'''

[tool.coverage.run]
omit = [
    'tests/*',
    '*_pb2.py',
    '*_pb2_grpc.py'
]

[tool.taskipy.tasks]
black = "black ."
mypy = "mypy . --ignore-missing-imports"
linter = "flake8 ."

And now it works with 1.1.0+.

So I don't know if we should close the issue, I think it's still a bug.

ChameleonTartu commented 4 years ago

I get a similar issue with the 1.1.2 version. link-python-common is a private repo project.

  • Installing link-python-common (1.0.0): Failed

  RuntimeError

  Unable to find installation candidates for link-python-common (1.0.0)

  at ~/.poetry/lib/poetry/installation/chooser.py:73 in choose_for
       69│             links.append(link)
       70│ 
       71│         if not links:
       72│             raise RuntimeError(
    →  73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link
       77│         chosen = max(links, key=lambda link: self._sort_key(package, link))

Example of toml file:

[[tool.poetry.source]]
name = "example-test-pypi"
url = "https://exmaple-url-pypi/pypi"
secondary=true

[tool.poetry]
name = "my-example-project-toml"
version = "0.1.0"
description = ""

[tool.poetry.dependencies]
python = "^3.7.7"
link-python-common = "^1.0.0"
xmljson = "^0.2.0"
returns = "^0.13.0"
typing_extensions = "3.7.4"
attrs = "19.1.0"
defusedxml = "^0.6.0"

[tool.poetry.dev-dependencies]
pytest = "^5.1"
mypy = "^0.770.0"
wemake-python-styleguide = "^0.14.0"
pre-commit = "^1.18"
requests = "^2.23.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Also, I set poetry config http-basic.example-test-pypi my_example_pypy_user my_example_pypi_password

Is there anything I can do to fix this? I can make a PR if you guide me on where and what should be fixed.

SharpEdgeMarshall commented 4 years ago

@ChameleonTartu same issue as yours

akoltsov-spoton commented 3 years ago

Workaround pip install -U 'poetry<1.1.0' failed with 1.1.0-1.1.4

ebr commented 3 years ago

Not sure if :+1: s are tracked on this issue, but I can reproduce this exactly. Poetry version 1.1.4.

sfontana commented 3 years ago

I experience the same problem with version 1.1.6. Downgrading to 1.0.10 works.

Teivaz commented 3 years ago

I have had the same issue. In my case I had sections in toml file in the following order:

[[tool.poetry.source]]
[tool.black]
[tool.pytest.ini_options]
[tool.poetry.scripts]
[tool.poetry]
[tool.poetry.dependencies]
[tool.poetry.dev-dependencies]
[build-system]

Managed to fix it by moving the tool.poetry.scripts after tool.poetry

xmnlab commented 1 year ago

I am having a similar issue.

This is the command I am executing (I am replacing here the original names by placeholders):

poetry add git+ssh://git@github.com:<org>/<repo>.git#master --verbose

 HangupException

  The remote server unexpectedly closed the connection.

  at ~/mambaforge/envs/myenv/lib/python3.8/site-packages/dulwich/protocol.py:214 in read_pkt_line
      210│ 
      211│         try:
      212│             sizestr = read(4)
      213│             if not sizestr:
    → 214│                 raise HangupException()
      215│             size = int(sizestr, 16)
      216│             if size == 0:
      217│                 if self.report_activity:
      218│                     self.report_activity(4, "read")

The following error occurred when trying to handle this error:

  GitProtocolError

  Repository not found.

  at ~/mambaforge/envs/myenv/lib/python3.8/site-packages/dulwich/client.py:1103 in fetch_pack
      1099│         with proto:
      1100│             try:
      1101│                 refs, server_capabilities = read_pkt_refs(proto.read_pkt_seq())
      1102│             except HangupException as exc:
    → 1103│                 raise _remote_error_from_stderr(stderr) from exc
      1104│             (
      1105│                 negotiated_capabilities,
      1106│                 symrefs,
      1107│                 agent,

$ poetry --version
Poetry (version 1.4.2)

DennisSchwartz commented 1 year ago

I ran into this using poetry 1.6.1 when changing the private repository to be supplemental and specifying the source of my private package as the private repo directly.

Turns out deleting poetry.lock and then running install has solved the problem for me.

SamuelSmets commented 9 months ago

Is there a decent fix for this issue. I'm using poetry 1.6.1 like @DennisSchwartz , but removing the poetry.lock file and running poetry install didn't fix the problem for me. I'm a bit lost now.

abn commented 8 months ago

@xmnlab your reported issue not related to the issue discussed here. It looks like you don't have the right keys configured or your network dropped out.

@SamuelSmets can you please provide more details or better yet maybe raise a new issue with specific details? I'd recommend you test first with the latest poetry version.

chdemko commented 7 months ago

I solve the issue for me by running:

poetry config keyring.enabled false

istvan-andrasi commented 5 months ago

I've found this open issue with the same problem, but upgrading from 1.7.1 to 1.8.3 fixed it for me.

python-poetry / poetry

Repository "private" does not exist | Since 1.1.0 #3114

Issue