conda / conda-lock

Lightweight lockfile for conda environments
https://conda.github.io/conda-lock/
Other
459 stars 102 forks source link

Non-optional dependency shows up as optional #434

Open paulius-hydrosat opened 1 year ago

paulius-hydrosat commented 1 year ago

Checklist

What happened?

Consider the following pyproject.toml file spec:

[tool.poetry.dependencies]
psycopg2 = ">=2.9.1"
python = "==3.11.*"

[tool.poetry.group.prod.dependencies]
dagster_postgres = { version = "==0.19.5", source = "pypi" }

After locking, psycopg2 appears as optional dependency:

- name: psycopg2
  version: 2.9.3
  manager: conda
  platform: linux-64
  dependencies:
    libgcc-ng: '>=12'
    libpq: '>=15.1,<16.0a0'
    python: '>=3.11,<3.12.0a0'
    python_abi: 3.11.*
  url: https://conda.anaconda.org/conda-forge/linux-64/psycopg2-2.9.3-py311h968e94b_2.conda
  hash:
    md5: fdbb71c126dfaa4149f026c31f536ca8
    sha256: 7bf7e2ba62f0559d92ce1eeff27bf8fecab66803e90ac411255340359642297f
  category: prod
  optional: true

It should not, it should be non-optional. I suppose this happens as dagster_postgres is optional and depends on psycopg2.

Conda Info

active environment : conda-lock-2
    active env location : /home/psarka/mambaforge/envs/conda-lock-2
            shell level : 3
       user config file : /home/psarka/.condarc
 populated config files : /home/psarka/mambaforge/.condarc
                          /home/psarka/.condarc
          conda version : 23.3.1
    conda-build version : not installed
         python version : 3.10.10.final.0
       virtual packages : __archspec=1=x86_64
                          __cuda=11.4=0
                          __glibc=2.35=0
                          __linux=5.19.0=0
                          __unix=0=0
       base environment : /home/psarka/mambaforge  (writable)
      conda av data dir : /home/psarka/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/psarka/mambaforge/pkgs
                          /home/psarka/.conda/pkgs
       envs directories : /home/psarka/mambaforge/envs
                          /home/psarka/.conda/envs
               platform : linux-64
             user-agent : conda/23.3.1 requests/2.28.2 CPython/3.10.10 Linux/5.19.0-43-generic ubuntu/22.04.2 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

Conda Config

==> /home/psarka/mambaforge/.condarc <==
channels:
  - conda-forge

==> /home/psarka/.condarc <==
env_prompt: {default_env}

Conda list

No response

Additional Context

This is the full pyproject.toml that I'm using:

[project]
name = "name"
version = "1.0"

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[tool.poetry.dependencies]
psycopg2 = ">=2.9.1"
python = "==3.11.*"

[tool.poetry.group.prod.dependencies]
dagster_postgres = { version = "==0.19.5", source = "pypi" }

[tool.conda-lock]
channels = ["conda-forge"]
platforms = ['linux-64']

[tool.conda-lock.dependencies]
pip = ">=23.1"
maresb commented 1 year ago

It seems like you're hitting #278, sorry about that.

Would it be feasible to move dagster_postgres to tool.conda-lock.dependencies?

paulius-hydrosat commented 1 year ago

Not feasible, but no worries, as a workaround I'll be installing all the categories. Sorry for missing #278, feel free to close this a duplicate.

maresb commented 1 year ago

Thanks a lot for the report. I wouldn't have expected you to find #278. I'm inclined to leave this open for visibility.

anibali commented 10 months ago

I believe that I'm facing a similar issue using environment.yml files with different categories. My main environment.yml file includes pyyaml as a dependency, but this gets marked as "optional" in the lockfile when I also include an "extras" environment.yml file which includes pyyaml as a transitive dependency.