Open pep-sanwer opened 1 year ago
Thanks a lot @pep-sanwer for the thoughtful comments! I like your suggestion, and your explanation of your usage. To put it in my own words:
[tool.poetry.dependencies]
.[tool.conda-lock.dependencies]
.But I do agree that favoring PyPI packages could lead to messy problems. In my typical workflow, I want to get as many packages from conda-forge as possible, and only then would I fall back to PyPI for those which are missing. So from that perspective I prefer the existing logic. (My biggest pain point ATM is that implicit dependencies aren't converted to conda-forge, but that's another issue...)
Another thing that bothers me is how we are giving Poetry preferential treatment, especially because it is so temperamental, and there are other great alternatives available. In particular, the source = "pypi"
key-value pair is not standards-compliant, so unless I'm mistaken it can only really be used with Poetry. I'd really like to see this somehow eliminated.
In any case, it might be interesting to add a flag to [tool.conda-lock]
to make [tool.poetry.dependencies]
(and even better the standards-compliant [project.dependencies]
) PyPI-only.
Thank you @maresb for taking the time to look into this.
I agree with you completely on keeping the current behavior - while exposing a flag parameter in [tool.conda-lock]
that would allow the user to specify default source behavior for [tool.poetry.dependencies]
.
To your point on favoring Poetry - I agree as well. Hopefully we are moving towards a more standards compliant [project.dependencies]
specification as an ecosystem. Until we get there, conda-lock
and poetry
have been a godsend.
Checklist
What is the idea?
My understanding is that as
pyproject.toml
(Poetry flavor) support currently stands, all dependencies default to conda channels unless explicitly stated assource = "pypi"
for that particular dependency. From what I understand, this includes dependencies listed in[tool.poetry.dependencies]
as well as[tool.conda-lock.dependencies]
.The feature suggestion here is change this default behavior, so that dependencies listed in
[tool.poetry.dependencies]
default to PyPI, i.e.source = "pypi"
without having to explicitly state it inpyproject.toml
, while[tool.conda-lock.dependencies]
behavior can remain the same, i.e. defaulting to conda channels unless explicitly stated otherwise inpyproject.toml
.This would enable workflows where you can
conda-lock
to solve an environment,conda-lock install
to install a reproducible environment, and thenpoetry add pandas
(for example) to add additional pyPI dependencies as the project evolves. As it currently stands, one would have to invokepoetry add pandas --source="pypi"
(for example) per additional pyPI dependency.Currently, the workflow I have landed upon is to lock and reproduce the environment with:
conda-lock --conda mamba -f pyproject.toml
conda-lock install --conda mamba --name test_env conda-lock.yml
poetry install
to update all pyPI packages from conda channels to their latest pyPIIf I am misunderstanding how additional dependencies should be added to a project using
conda-lock
&pyproject.toml
or how conda-lock should be utilized with Poetry, please correct me.An additional question would also be, does
conda-lock
default toconda-lock --conda mamba
if the onlyconda
installed is frommamba-forge
?Appreciative of this wonderful tool!
Why is this needed?
No response
What should happen?
No response
Additional Context
No response