python-poetry / poetry-plugin-export

Poetry plugin to export the dependencies to various formats
MIT License
238 stars 53 forks source link

Missing ``--extra-index-url`` lines in ``requirements.txt`` for second primary package source #269

Closed nsoranzo closed 6 months ago

nsoranzo commented 6 months ago

For a project I work on, we have a different package source configured as primary (since poetry deprecated default) with PyPI as second primary, as recommended in https://python-poetry.org/docs/repositories/ .

This is the relevant section of our pyproject.toml:

[[tool.poetry.source]]
name = "galaxyproject"
url = "https://wheels.galaxyproject.org/simple"
priority = "primary"

[[tool.poetry.source]]
name = "PyPI"
priority = "primary"

When exporting with poetry export -f requirements.txt --without-hashes --output requirements.txt the output file starts with just:

--index-url https://wheels.galaxyproject.org/simple

so when trying to install the project dependencies from this file, it fails with:

Looking in indexes: https://wheels.galaxyproject.org/simple
...
ERROR: Could not find a version that satisfies the requirement a2wsgi==1.10.4 (from versions: none)
ERROR: No matching distribution found for a2wsgi==1.10.4

Before the merge of https://github.com/python-poetry/poetry-plugin-export/pull/263 , the file started with:

--extra-index-url https://wheels.galaxyproject.org/simple

and it worked fine. I think the proper solution is to also add for each primary after the first an --extra-index-url URL line, e.g. adding the following line fixes the issue for us:

--extra-index-url https://pypi.python.org/simple
radoering commented 6 months ago

I think the proper solution is to also add for each primary after the first an --extra-index-url URL line

Actually, that's what is done. See https://github.com/python-poetry/poetry-plugin-export/blob/main/tests/test_exporter.py#L1792

Probably, there is something special about PyPI that is not handled correctly.

nsoranzo commented 6 months ago

Thanks for the quick reply @radoering ! I've done a quick test by amending the second primary package source to not look like PyPI:

[[tool.poetry.source]]
name = "foo"
url = "https://pypi.python.org/simple"
priority = "primary"

and after running poetry lock --no-update and poetry export -f requirements.txt --without-hashes --output requirements.txt the output file starts with:

--extra-index-url https://pypi.python.org/simple
--index-url https://wheels.galaxyproject.org/simple

which seems to also not work in this order (i.e. --index-url should come before --extra-index-url).

radoering commented 6 months ago

e.g. adding the following line fixes the issue for us:

--extra-index-url https://pypi.python.org/simple

I chose a different approach in #270. Since pip does not care about order by design (see pypa/pip#8606 for details), it does not matter which source is the --index-url and which is the --extra-index-url and since PyPI is the default --index-url, I chose to just not set --index-url if PyPI is among the sources. (That's what it was before the last release anyway.)

which seems to also not work in this order (i.e. --index-url should come before --extra-index-url).

That's actually a pip bug. See #149. But since it is easy to make sure that --index-url comes first, I also changed it in #270.