search

rapidsai / dask-cuda

Utilities for Dask and CUDA interactions
https://docs.rapids.ai/api/dask-cuda/stable/
Apache License 2.0
292 stars 93 forks source link

Conda-forge packaging dependencies #1208

Open ericpre opened 1 year ago

ericpre commented 1 year ago

I have opened a pull request to the conda-forge feedstock of dask-cuda to avoid installing unnecessary dependency (pyarrow, bokeh) by removing dask and keeping dask-core: https://github.com/conda-forge/dask-cuda-feedstock/pull/26. As pyarrow and bokeh are not defined in https://github.com/rapidsai/dask-cuda/blob/e728eb6ae2feca50b58f131f9a1c7fd8451b0801/pyproject.toml#L18-L26

they will not be installed when installing from pypi but they will when installed from conda-forge.

@jakirkham suggested to open an issue to discuss here. As I understand, this is a conda-forge packaging only concern but it doesn't hurt opening an issue to discuss it!

What do you think?

pentschev commented 1 year ago

I agree with this change. None of these dependencies are really necessary and making them optional makes the most sense.

jakirkham commented 1 year ago

Currently we are using pyproject.toml as the source of truth for both wheels & conda packages, which causes this confusion on the dask dependency

Think we should move to dependencies.yaml and borrow cuDF's approach to encoding the Dask dependency

If we make that change, then it will be straightforward to have only dask-core in the conda case and dask in the wheel case

ericpre commented 10 months ago

What is the status of this issue?

jakirkham commented 10 months ago

These are the currently list dependencies

https://github.com/rapidsai/dask-cuda/blob/897f0b6af101fc8bed24ab4c636ca06d297496d3/pyproject.toml#L18-L26

Updating the recipe in conda-forge to a more recent version would allow for cleanup to occur. There is some work needed in conda-forge to update dask-cuda ( https://github.com/conda-forge/dask-cuda-feedstock/pull/29#discussion_r1424672350 )