Open jameslamb opened 4 months ago
@bdice @trxcllnt @KyleFromNVIDIA I know we reached a tentative conclusion on a call earlier today that this should be unnecessary, but I think it'd still be useful.
I wrote this up to hopefully clarify the exact design question(s) and what rapids-build-backend
currently can / cannot do.
Description
Created from @bdice's suggestion at https://github.com/rapidsai/cudf/pull/15245#discussion_r1617850787
RAPIDS projects'
dependencies.yaml
files should be modified such that the no-CUDA-version lists contain the set of dependencies needed when targeting CUDA 12 (but without suffixes like-cu12
).For example, consider the following from
cudf
If you run
rapids-dependency-file-generator --output requirements
against that without acuda
value provided, it'll produce something like this.Those fallback
dependencies.yaml
blocks should be modified such that it instead generates something like this.With a change like this:
Benefits of this work
pyproject.toml
/requirements*.txt
checked in source control will look more like the dependencies for building the library against the latest supported version of CUDAcudf-cu12
), removes some need to patch requirements filesAcceptance Criteria
For every project:
rapids-dependency-file-generator --output requirements
or--output pyproject
for any keys that produce those outputs produces a list of dependencies for CUDA 12 (without suffixes like-cu12
)Approach
Some projects might need 0 changes... this situation where projects depend on different libraries (not just different versions / suffixed names) across CUDA major versions isn't universal.
For the others that do need changes, modify them following the
cudf
example from above.Notes
Isn't
rapids-build-backend
supposed to solve this?As of today, it doesn't.
It supports an option,
disable-cuda
, which iftrue
is interpreted as "do not try to guess the target CUDA version... leave project name and dependency names unsuffixed, and use the fallback matrices independencies.yaml
"(rapids-build-backend-docs.
Those things could be decomposed into separate settings, so that you could for example do something like this with a project using
rapids-build-backend
:(or set any of those via environment variables instead of
-C
flags passed to the builder)Related conversation: https://github.com/rapidsai/cudf/pull/15245#discussion_r1617853918.
But in the current state of
rapids-build-backend
, that isn't supported.Since the
dependencies.yaml
updates should be low-effort and wouldn't preclude something like therapids-build-backend
API changes, and since they have at least one other benefit (pyproject.toml
/requirements.txt
in source control more closely matching what'll be pulled in when targeting the latest CUDA version) I think it'd be worth making thesedependencies.yaml
changes regardless of any decision about how to modifyrapids-build-backend
to support DLFW-style builds.