Description

It's common across RAPIDS to build a wheel in one CI job and then use it in another one (for example, building librmm and then using that in a later job that builds rmm).

69 documents some cases where overly-flexible approaches for passing those wheels between jobs led to CI failing to catch broken wheels.

It also documents an approach that was taken to prevent that... in pseudo-code:

echo 'librmm-cu12 @ file://opt/downloads/dist/librmm_cu12.whl' > constraints.txt

PIP_CONSTRAINT=${PWD}/constraints.txt \
  pip wheel .

In https://github.com/rapidsai/kvikio/pull/397/files#r1657898551, @jakirkham recommended considering instead doing something like the following:

pip wheel \
  --no-index \
  --find-links /opt/download/dist/ \
  .

To close this issue, investigate that alternative approach and roll it out if it's found to be preferable.

Benefits of this work

Similar to #69, but with:

lower risk of silent issues caused by mistakes in generating a constraints.txt file
reduced risk of the portability issues that come with using file-based constraints (line endings, encodings, slashes in paths, etc.)

Acceptance Criteria

this approach has been investigated
if the approach is deemed preferable to the one from #69, it's been rolled out to all affected RAPIDS repos

Approach

Test the approach by following something like "how I tested this" from https://github.com/rapidsai/rmm/pull/1586

Notes

Opened this to defer investigation it further in #69, but I (@jameslamb) want to note one reservation for whenever this is picked up.

Using --no-index --find-links ${DIR} probably means that ${DIR} needs to contain all build dependencies (e.g. scikit-build-core, Cython, etc.)... so scripts would have to somehow provide for filling that directory with those, e.g. with pip download.

If that's true, it might just be simpler to completely stop using build isolation (e.g. stop passing --no-build-isolation) in wheel build jobs and set up the build environment with e.g. pip install or pip download (and then passing the other wheels from CI via constraints like --constraint /opt/download/dist/*.whl).

rapidsai / build-planning