[[tool.poetry.source]] setting is ignored

zillionare commented 3 years ago

[x ] I am on the latest Poetry version.
[ x] I have searched the issues of this repo and believe that this is not a duplicate.
[ ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name:
Poetry version: 1.1.4
Link of a Gist with the contents of your pyproject.toml file:

Issue

I'm in china mainland and speed to access pypi is very slow, so I have added tool.poetry.source to a local source:

[[tool.poetry.source]]
name = "ali"
url = "https://mirrors.aliyun.com/pypi/simple/"

However, this one works for my one project yesterday, it doesn't work for my another project, checked today.

When it works, I found poetry will check both pypi and this local source for index, but only download package from local source, this can enhance speed a lot. However, it seems not work today, it always check index and download package from pypi, the local source is totally ignored.

if poetry is downloading from local source(which is ali in the case), it'll show source's name before ':'. it's pypi here since it downloads from pypi.

I have recreated my virtual env and delete pypoetry's cache.

sinoroc commented 3 years ago

Maybe you need to disable PyPI as a source?

https://python-poetry.org/docs/repositories/#disabling-the-pypi-repository

[[tool.poetry.source]]
name = "ali"
url = "https://mirrors.aliyun.com/pypi/simple/"
default = true

zillionare commented 3 years ago

Thanks! @sinoroc

No luck either. Now I have to setup a proxy, and I see all the traffic goes to pypi:

it works, but still too slow, compares to a local mirror pypi

sinoroc commented 3 years ago

That seems strange to me.

Are there maybe still references to pypi.org in the lockfile? If possible delete the lockfile and try again.
Are you maybe hitting the issue https://github.com/python-poetry/poetry/issues/3249 ?

GooseYArd commented 3 years ago

I'm having the same issue with a twist- I have multiple poetry-based projects on a single machine with nearly identical pyproject.toml files, all but one work successfully using the private repo. After bisecting the pyproject.toml file, I realized that the addition of "[tool.poetry.scripts]" to the bad module is what triggers this issue, even if there are no console scripts defined in the block.

I haven't had time to debug any further but figured I'd mention this here in case someone is able to get to before I do.

GooseYArd commented 3 years ago

after some debugging, I found that if "[[tool.poetry.source]]" precedes "[tool.poetry.scripts]" in pyproject.toml, then as early as PyProjectTOML.data(), self._data["tool"]["poetry"] will be absent a "source" key. However, if I move tool.poetry.source below tool.poetry.scripts in the file, this problem doesn't occur.

I'm not sure if the issue is my ignorance of toml or an issue with tomlkit, but will do a little more debugging to see if I can spot the root cause.

kevinhuang28 commented 3 years ago

I ran into the same issue on 1.1.5 without tool.poetry.scripts section. I had tool.poetry.source after tool.poetry and nothing was being picked up. I ran into this thread and tried moving tool.poetry.source to the end of the file and it worked!

Example of what I had:

[tool.poetry]
name = "abcd"
version = "1.1.1"
description = "abcd"
authors = ["ABCD <abcd@abcd.com>"]

[[tool.poetry.source]]
name = "fury"
url = "https://pypi.fury.io/abcd/"
secondary = true

[tool.poetry.dependencies]
xyz = { version = "1.1.1", source = "fury" }
ddtrace = "^0.40.0"
django = "^3.0.7"
django-environ = "^0.4.5"
django-extensions = "^2.2.9"
django-redis = "^4.12.1"
fastavro = "^1.3.2"
glom = "^20.5.0"
google-cloud-pubsub = "^1.6.0"
google-cloud-storage = "^1.29.0"
gunicorn = "^20.0.4"
pendulum = "^2.1.1"
psycopg2 = "^2.8.6"
python = "^3.7"
python-json-logger = "^0.1.11"
python-redis-lock = {version = "^3.7.0", extras = ["django"]}
requests = "^2.24.0"
sentry-sdk = "^0.16.1"
python-slugify = "^4.0.1"

[tool.poetry.dev-dependencies]
pytest = "*"
pytest-django = "*"
pytest-tldr = "*"
pytest-sugar = "*"
pytest-cov = "*"
flake8 = "^3.8.3"
black = "^19.10b0"
bandit = "^1.6.2"
pre-commit = "^2.5.1"
pytest-watch = "^4.2.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

zillionare commented 3 years ago

As I'm using poetry more and more, I might found clue leading to my issues (I haven't been through this for months).

At time when the issue reported, I think we're going though pip's upgrade; and there's sync issue between central repo (pypi) and its mirrors (for example, aliyun in my case). According to my observation, the sync may take several days.

As more and more python libraries manage dependancy with poetry, I found now dep resolution time is decreased, so a second source is not required (it's worth some time to resolve deps use central repo, other than mirrors).

If you still experiece slow resolution, maybe you could try resolve your project.toml on a machine located at us cloud service. I did that several times months ago.

close this one since I didn't expierence issues anymore, and I think it's probably resolved

github-actions[bot] commented 9 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

python-poetry / poetry

[[tool.poetry.source]] setting is ignored #3474

Issue