search

conda-forge / openmpi-feedstock

A conda-smithy repository for openmpi.
BSD 3-Clause "New" or "Revised" License
9 stars 25 forks source link

Erroneous dependency on `cudatoolkit` #78

Closed leofang closed 3 years ago

leofang commented 3 years ago

Looks like the build time dependency on cudatoolkit leaks out to runtime...😞

$ conda install -c conda-forge mpi4py openmpi
Collecting package metadata (current_repodata.json): done
Solving environment: - 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - conda-forge/linux-64::cutensor==1.2.2.5=h96e36e3_2
done

==> WARNING: A newer version of conda exists. <==
  current version: 4.8.3
  latest version: 4.9.2

Please update conda by running

    $ conda update -n base -c defaults conda

## Package Plan ##

  environment location: /home/leofang/miniconda3/envs/cupy_cuda112_dev

  added / updated specs:
    - mpi4py
    - openmpi

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cudatoolkit-11.0.3         |       h15472ef_7       952.4 MB  conda-forge
    mpi4py-3.0.3               |   py37h3e1c8c0_5         700 KB  conda-forge
    openmpi-4.1.0              |       h9b22176_1         3.9 MB  conda-forge
    ------------------------------------------------------------
                                           Total:       957.0 MB

The following NEW packages will be INSTALLED:

  cudatoolkit        conda-forge/linux-64::cudatoolkit-11.0.3-h15472ef_7
  mpi                conda-forge/linux-64::mpi-1.0-openmpi
  mpi4py             conda-forge/linux-64::mpi4py-3.0.3-py37h3e1c8c0_5
  openmpi            conda-forge/linux-64::openmpi-4.1.0-h9b22176_1
jakirkham commented 3 years ago

Is this still the case if one changes...

https://github.com/conda-forge/openmpi-feedstock/blob/df24491446815f5c597ba93c4147e4d1e1104957/recipe/meta.yaml#L28-L29

...to...

      ignore_run_exports:
        - cudatoolkit

?

Wondering if ignore_run_exports_from might not be working as expected, but perhaps ignore_run_exports would

leofang commented 3 years ago

Could it be a bug in conda? I just tested on another machine that also has a GPU and an older conda, and it doesn't show up:

$ conda install -c conda-forge mpi4py openmpi
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 4.7.12
  latest version: 4.9.2

Please update conda by running

    $ conda update -n base -c defaults conda

## Package Plan ##

  environment location: /home/leofang/miniconda3/envs/cupy_dev_cuda11

  added / updated specs:
    - mpi4py
    - openmpi

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2020.12.5  |       ha878542_0         137 KB  conda-forge
    certifi-2020.12.5          |   py37h89c1867_1         143 KB  conda-forge
    mpi4py-3.0.3               |   py37h3e1c8c0_4         703 KB  conda-forge
    openssl-1.1.1i             |       h7f98852_0         2.1 MB  conda-forge
    ------------------------------------------------------------
                                           Total:         3.1 MB

The following NEW packages will be INSTALLED:

  mpi                conda-forge/linux-64::mpi-1.0-openmpi
  mpi4py             conda-forge/linux-64::mpi4py-3.0.3-py37h3e1c8c0_4
  openmpi            pkgs/main/linux-64::openmpi-4.0.2-hb1b8bf9_1

The following packages will be UPDATED:

  ca-certificates                      2020.6.20-hecda079_0 --> 2020.12.5-ha878542_0
  certifi                          2020.6.20-py37hc8dfbb8_0 --> 2020.12.5-py37h89c1867_1
  openssl                                 1.1.1g-h516909a_1 --> 1.1.1i-h7f98852_0

But oddly, all CI tests do not have this problem either...

jakirkham commented 3 years ago

Certainly possible, in which case my suggestion above may not help

jakirkham commented 3 years ago

Are you able to isolate the Conda version that has the bug?

leofang commented 3 years ago

The CI is running on the latest conda, right?

jakirkham commented 3 years ago

Looks like it

leofang commented 3 years ago

I couldn't reproduce this consistently, closing this for now.