jameslamb
commented
5 months ago @bdice drew my attention to these global pins in conda-forge-pinning-feestock:
fmt:
- '10'
spdlog:
- '1.12'I think the fmt one shouldn't be a problem for this migration, but the spdlog one will. There's an ongoing conda-forge migration to spdlog==1.13.0, but I don't know how to estimate how long that'll take: https://conda-forge.org/status/migration/spdlog113.
I think we should NOT update any RAPIDS packages to depending on the spdlog==1.13.0 conda-forge package until that conda-forge migration to that version is done. Does that seem right @jakirkham @bdice ?
References:
- "Pinned dependencies" (conda-forge docs)
- https://github.com/conda-forge/conda-forge-pinning-feedstock
bdice
kkraus14
Description
RAPIDS projects that need
fmtandspdloguserapids-cmaketo find it, via functions likerapids_cpm_fmt()(docs link).rapids-cmakeis currently carrying patches to both of those libraries, and as a result always downloads sources of them. (see https://github.com/rapidsai/rapids-cmake/pull/525 for details).Those patches have since been upstreamed and made it into official releases of those projects (in their main source control and on
conda-forge). This proposes upgrading to those new versions across RAPIDS and dropping the patches inrapids-cmake.Benefits of this work
condabuilds across many RAPIDS projectsrmmto ensure future such conflicts raise a loud errorrapids-cmakeand therefore the need to reason about them in future upgradesAcceptance Criteria
rapids-cmakeis pulling infmt >= 11.0.1rapids-cmakeis pulling inspdlog >= 1.14fmtandspdloginrmm's conda build CI logsApproach
On a branch, modify
rapids-cmake/cpm/versions.json(code link) such that:fmtandspdlogare bumped to these new onesFollow "Overriding RAPIDS CMake" (rapids-cmake docs) to point builds of at least
rmm,cudf, andraftat your branch ofrapids-cmakewith these changes. Also modify those projects'dependencies.yamland/or conda recipemeta.yamlto change their version constraints onfmtandspdlog.NOTE: this is a great task for
rapids-reviser(https://github.com/rapidsai/rapids-reviser/tree/main/examples/fmt-10-spdlog-1.12).Confirm that CI succeeds and that the correct versions of
fmtandspdlogare being pulled in.For
rmm, re-use this PR for that purpose: https://github.com/rapidsai/rmm/pull/1508. If this change also results in there being 0 remaining conda clobbering warnings inrmm, leave in theconda config --set path_conflict preventin that PR's build scripts, so we'll be alerted by a CI failure if something around this changes in the future.Notes
What makes you confident that these patches can be removed?
From https://github.com/rapidsai/build-planning/issues/54#issuecomment-2072893399
fmt's most recent release was11.0.2, so that patch we're carrying around for the10.1.1version could be dropped.rapids-cmake: (fmt/fix_10_1_1_version.diff)v10.2.1release on conda-forge removing a similar patch: https://github.com/conda-forge/fmt-feedstock/pull/48The
spdlogpatch we've been carrying around has been upstreamedrapids-cmake: (spdlog/nvcc_constexpr_fix.diff)v1.13.0of the conda-forge recipe is up: https://github.com/conda-forge/spdlog-feedstock/pull/61What are "clobber warnings" and how does this help with them?
As described in #54 and https://github.com/rapidsai/rmm/pull/1508, conda builds of
rmmand other RAPIDS packages downstream of it are emitting dozens of warnings like these:These are saying that the
librmmconda package contains files whose paths exactly conflict with those from thefmtandspdlogpackages fromconda-forge. When such packages are installed together, one will overwrite the other, which could lead to runtime issues.Removing the patches in
rapids-cmakemakes it more likely that builds will find the files they need from thefmtandspdlogconda-forge packages already present in the build environment, and therefore not vendor them, and therefore not ship a copy that causes conflicts.