Optional Dependencies Have Different Behavior in Conda Lock vs Poetry

[srilman]

Checklist

• [X] I added a descriptive title
• [X] I searched open reports and couldn't find a duplicate

What happened?

I noticed this odd combination that Poetry seems to handle in one way and conda-lock does in another. Assume we have the following pyproject.toml

Situtation 1

[tool.poetry]
name = "conda-lock-test-poetry"
version = "0.0.1"

[tool.poetry.dependencies]
toml = ""
tomlkit = { version = "", optional = true }

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

[tool.conda-lock]
platforms = ["osx-arm64"]
channels = ["conda-forge"]

This is a valid specification in poetry. However, if we were to lock and install this file with Poetry, it would only install the toml package and it's dependencies in the virtualenv. It seems like optional dependencies that aren't part of any extra category can never be installed by Poetry (see https://stackoverflow.com/questions/60971502/python-poetry-how-to-install-optional-dependencies).

Conda lock has slightly different behavior; it will lock both toml and tomlkit in the lockfile under the main category, but the lockfile will specify that toml is required and tomlkit is optional. However, when installing from the lockfile, tomlkit will be included in the resulting env by default, and I don't think there is any way to exclude it.

Situtation 2

Furthermore, it seems like dev dependencies (specified in tool.poetry.dev-dependencies section) can and should never be optional (see https://stackoverflow.com/questions/60971502/python-poetry-how-to-install-optional-dependencies). Poetry will completely ignore them, but conda-lock will still treat them like dev dependencies and they will be optional. If the user specified optional = false they will be set as required in the lockfile.

Conda Info

N/A

Conda Config

N/A

Conda list

N/A

Additional Context

Suggested Fix for Situation 1

We can do one of the following: 1) Include optional dependencies, but in conda-lock, mark the category as main and required (assuming in general that if the category is main, it is a required dependency Should we throw a warning indicating that the optional flag in the pyproject.toml was ignored? 2) Skip optional dependencies (with or without warning) 3) Throw an error if invalid optional dependencies are found 4) Create a new category for these

I would say lets go with option (1) and throw a warning.

Suggested Fix for Situation 2

We can do one of the following:

• Include optional dependencies, but in conda-lock, mark the category as dev and optional. Warning or no warning?
• Skip optional dependencies (with or without warning)
• Throw an error if invalid optional dependencies are found
• Create a new category for these

Again, I would say lets go with option (1) and throw a warning.