Closed bdice closed 6 months ago
To solve https://github.com/rapidsai/build-planning/issues/22, I'm thinking we'll do this kind of thing in each RAPIDS repo CI test script:
rapids-logger "Downloading artifacts from previous jobs"
CPP_CHANNEL=$(rapids-download-conda-from-s3 cpp)
PYTHON_CHANNEL=$(rapids-download-conda-from-s3 python)
rapids-dependency-file-generator \
--output conda \
--file_key test_python \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" \
--prepend-channels "${CPP_CHANNEL};${PYTHON_CHANNEL}" | tee env.yaml
rapids-mamba-retry env create --force -f env.yaml -n test
We would modify the test_cpp
file key to include a dependency list containing libcuml
and libcuml-tests
. We would also modify the test_python
file key to include a dependency list containing libcuml
and cuml
. Then this environment would be equivalent to the conda-merge
output I was using in https://github.com/rapidsai/cuml/pull/5781.
:tada: This PR is included in version 1.9.0 :tada:
The release is available on:
Your semantic-release bot :package::rocket:
This adds an option
--prepend-channels "my_channel;my_other_channel"
to therapids-dependency-file-generator
CLI. This will allow us to userapids-dependency-file-generator
with local channels containing PR artifacts fetched in CI workflows. If we combine this feature with some small changes independencies.yaml
, we will be able to generate the entire CI environment in one pass rather than generating an environment with test dependencies and then installing the packages (e.g.libcuml
andlibcuml-tests
) in a separate step.This option is modeled after
conda config --prepend channels new_channel
(reference).See original proposal in https://github.com/rapidsai/cuml/pull/5781#issuecomment-1958498274 (this PR is slightly improved from that proposal).
This is a partial solution for https://github.com/rapidsai/build-planning/issues/22.