rapidsai / build-planning

Tracking for RAPIDS-wide build tasks
https://github.com/rapidsai
0 stars 1 forks source link

Align conda and wheel building workflows #44

Open vyasr opened 1 month ago

vyasr commented 1 month ago

Historically our conda and wheel GHA workflow scripts have looked fairly different for a number of reasons. However, with #33 many of the fundamental distinctions will no longer exist because wheels will also have separate build steps for C++ and Python builds. As a result, we should invest in aligning our workflows as much as possible so as to reduce maintenance costs going forward. Some changes that we ought to make:

I will update this list as more ideas come to mind.

vyasr commented 1 month ago

A separate but related topic that may make sense to address somewhat in parallel is finding a way to make some subsets of the scripts reusable. Some work was previously done to make the CI test scripts usable in other environments (e.g. https://github.com/rapidsai/raft/pull/2165), so when aligning conda/wheels CI and finding ways to maximize shared code (e.g. both calling the same test_* scripts) it might also make sense to see if we can make sure build and test scripts could be used in other environments, such as devcontainers. However, in an ideal world build scripts would be trivial (i.e. just cmake -S . -B build && cmake --build build, or python -m pip install ..., so there might not be much to do for build scripts (and test scripts have already been handled as mentioned above).

jameslamb commented 1 month ago

The rapids-wheels-anaconda tool will need to be modified to support upload of cpp wheels.

Put up a proposal for this in https://github.com/rapidsai/gha-tools/pull/105