Open vyasr opened 1 month ago
A separate but related topic that may make sense to address somewhat in parallel is finding a way to make some subsets of the scripts reusable. Some work was previously done to make the CI test scripts usable in other environments (e.g. https://github.com/rapidsai/raft/pull/2165), so when aligning conda/wheels CI and finding ways to maximize shared code (e.g. both calling the same test_*
scripts) it might also make sense to see if we can make sure build and test scripts could be used in other environments, such as devcontainers. However, in an ideal world build scripts would be trivial (i.e. just cmake -S . -B build && cmake --build build
, or python -m pip install ...
, so there might not be much to do for build scripts (and test scripts have already been handled as mentioned above).
The rapids-wheels-anaconda tool will need to be modified to support upload of cpp wheels.
Put up a proposal for this in https://github.com/rapidsai/gha-tools/pull/105
Historically our conda and wheel GHA workflow scripts have looked fairly different for a number of reasons. However, with #33 many of the fundamental distinctions will no longer exist because wheels will also have separate build steps for C++ and Python builds. As a result, we should invest in aligning our workflows as much as possible so as to reduce maintenance costs going forward. Some changes that we ought to make:
RAPIDS_PY_WHEEL_NAME
. In the PRs for #33 we're currently abusingRAPIDS_PY_WHEEL_NAME
to handle the CUDA version, so we need to start adding it for wheels before we can get rid of that variable.rapids-download-conda-from-s3
automates choosing the output directory, whilerapids-download-wheels-from-s3
requires that the caller specify it. We should update the wheel tool to automate that too.build_wheel_*
, whereas conda is justbuild_cpp.sh
etc. That is an artifact of a time when conda was our only produced artifact.rapids-wheels-anaconda
tool will need to be modified to support upload of cpp wheels.I will update this list as more ideas come to mind.