search

conda-forge / cuda-feedstock

A conda-smithy repository for cuda.
BSD 3-Clause "New" or "Revised" License
2 stars 8 forks source link

Fix package clobbering #52

Open jakirkham opened 21 hours ago

jakirkham commented 21 hours ago

Some packages appear to install content (like license files) in the $PREFIX without naming them or placing them in a way to avoid clobbering between packages (for example: https://github.com/conda-forge/cuda-nvvp-feedstock/pull/14#issuecomment-2457629212 )

We should work to remove any clobbering issues between CUDA Toolkit packages

When it comes to license files specifically, we already make sure to include these in all packages with about/license_file (for example). So there is no need to duplicate this effort. So with license files specifically, we should not copy them to the $PREFIX, which should fix that particular kind of clobbering

More generally we may just want to adapt scripts to only copy directories

If we need to copy over any files, we likely will need to do so in a clearly specified way

Other clobbering issues can be handled on a case by case basis

jakirkham commented 21 hours ago

To help identify which packages have clobbering issues, tried creating an environment with the cuda package and clobbering warnings enabled

Here is the full output:

``` $ CONDA_PATH_CONFLICT="warn" conda create -n cuda cuda Channels: - conda-forge Platform: linux-aarch64 Collecting package metadata (repodata.json): done Solving environment: done ## Package Plan ## environment location: /opt/conda/envs/cuda added / updated specs: - cuda The following NEW packages will be INSTALLED: _openmp_mutex conda-forge/linux-aarch64::_openmp_mutex-4.5-2_gnu alsa-lib conda-forge/linux-aarch64::alsa-lib-1.2.12-h68df207_0 attr conda-forge/linux-aarch64::attr-2.5.1-h4e544f5_1 binutils conda-forge/linux-aarch64::binutils-2.43-hf1166c9_2 binutils_impl_lin~ conda-forge/linux-aarch64::binutils_impl_linux-aarch64-2.43-h4c662bb_2 binutils_linux-aa~ conda-forge/linux-aarch64::binutils_linux-aarch64-2.43-hf1166c9_2 bzip2 conda-forge/linux-aarch64::bzip2-1.0.8-h68df207_7 c-compiler conda-forge/linux-aarch64::c-compiler-1.8.0-h6561dab_1 ca-certificates conda-forge/linux-aarch64::ca-certificates-2024.8.30-hcefe29a_0 cuda conda-forge/noarch::cuda-12.6.2-ha804496_0 cuda-cccl_linux-a~ conda-forge/noarch::cuda-cccl_linux-aarch64-12.6.77-h579c4fd_0 cuda-command-line~ conda-forge/linux-aarch64::cuda-command-line-tools-12.6.2-h8af1aa0_0 cuda-compiler conda-forge/noarch::cuda-compiler-12.6.2-hbad6d8a_0 cuda-crt-dev_linu~ conda-forge/noarch::cuda-crt-dev_linux-aarch64-12.6.77-h579c4fd_1 cuda-crt-tools conda-forge/linux-aarch64::cuda-crt-tools-12.6.77-h579c4fd_1 cuda-cudart conda-forge/linux-aarch64::cuda-cudart-12.6.77-h3ae8b8a_0 cuda-cudart-dev conda-forge/linux-aarch64::cuda-cudart-dev-12.6.77-h3ae8b8a_0 cuda-cudart-dev_l~ conda-forge/noarch::cuda-cudart-dev_linux-aarch64-12.6.77-h3ae8b8a_0 cuda-cudart-static conda-forge/linux-aarch64::cuda-cudart-static-12.6.77-h3ae8b8a_0 cuda-cudart-stati~ conda-forge/noarch::cuda-cudart-static_linux-aarch64-12.6.77-h3ae8b8a_0 cuda-cudart_linux~ conda-forge/noarch::cuda-cudart_linux-aarch64-12.6.77-h3ae8b8a_0 cuda-cuobjdump conda-forge/linux-aarch64::cuda-cuobjdump-12.6.77-h05609ea_0 cuda-cupti conda-forge/linux-aarch64::cuda-cupti-12.6.80-h5101a13_0 cuda-cupti-dev conda-forge/linux-aarch64::cuda-cupti-dev-12.6.80-h3ae8b8a_0 cuda-cuxxfilt conda-forge/linux-aarch64::cuda-cuxxfilt-12.6.77-h05609ea_1 cuda-driver-dev conda-forge/linux-aarch64::cuda-driver-dev-12.6.77-h3ae8b8a_0 cuda-driver-dev_l~ conda-forge/noarch::cuda-driver-dev_linux-aarch64-12.6.77-h3ae8b8a_0 cuda-gdb conda-forge/linux-aarch64::cuda-gdb-12.6.77-h704f0fb_1 cuda-libraries conda-forge/linux-aarch64::cuda-libraries-12.6.2-h8af1aa0_0 cuda-libraries-dev conda-forge/linux-aarch64::cuda-libraries-dev-12.6.2-h8af1aa0_0 cuda-nvcc conda-forge/linux-aarch64::cuda-nvcc-12.6.77-ha346c71_2 cuda-nvcc-dev_lin~ conda-forge/noarch::cuda-nvcc-dev_linux-aarch64-12.6.77-h4310d6a_1 cuda-nvcc-impl conda-forge/linux-aarch64::cuda-nvcc-impl-12.6.77-h614329b_1 cuda-nvcc-tools conda-forge/linux-aarch64::cuda-nvcc-tools-12.6.77-h614329b_1 cuda-nvcc_linux-a~ conda-forge/linux-aarch64::cuda-nvcc_linux-aarch64-12.6.77-hf9514e2_2 cuda-nvdisasm conda-forge/linux-aarch64::cuda-nvdisasm-12.6.77-h5101a13_1 cuda-nvml-dev conda-forge/linux-aarch64::cuda-nvml-dev-12.6.77-h5101a13_1 cuda-nvprune conda-forge/linux-aarch64::cuda-nvprune-12.6.77-h05609ea_1 cuda-nvrtc conda-forge/linux-aarch64::cuda-nvrtc-12.6.77-h5101a13_0 cuda-nvrtc-dev conda-forge/linux-aarch64::cuda-nvrtc-dev-12.6.77-h3ae8b8a_0 cuda-nvtx conda-forge/linux-aarch64::cuda-nvtx-12.6.77-h5101a13_0 cuda-nvvm-dev_lin~ conda-forge/noarch::cuda-nvvm-dev_linux-aarch64-12.6.77-h579c4fd_1 cuda-nvvm-impl conda-forge/linux-aarch64::cuda-nvvm-impl-12.6.77-h614329b_1 cuda-nvvm-tools conda-forge/linux-aarch64::cuda-nvvm-tools-12.6.77-h614329b_1 cuda-profiler-api conda-forge/linux-aarch64::cuda-profiler-api-12.6.77-h16bee8c_0 cuda-runtime conda-forge/noarch::cuda-runtime-12.6.2-ha804496_0 cuda-sanitizer-api conda-forge/linux-aarch64::cuda-sanitizer-api-12.6.77-h4177691_0 cuda-toolkit conda-forge/noarch::cuda-toolkit-12.6.2-ha804496_0 cuda-tools conda-forge/linux-aarch64::cuda-tools-12.6.2-h8af1aa0_0 cuda-version conda-forge/noarch::cuda-version-12.6-h7480c83_3 cuda-visual-tools conda-forge/linux-aarch64::cuda-visual-tools-12.6.2-h8af1aa0_0 cxx-compiler conda-forge/linux-aarch64::cxx-compiler-1.8.0-heb6c788_1 dbus conda-forge/linux-aarch64::dbus-1.13.6-h12b9eeb_3 expat conda-forge/linux-aarch64::expat-2.6.3-h5ad3122_0 font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0 font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0 font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0 font-ttf-ubuntu conda-forge/noarch::font-ttf-ubuntu-0.83-h77eed37_3 fontconfig conda-forge/linux-aarch64::fontconfig-2.15.0-h8dda3cd_1 fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0 fonts-conda-forge conda-forge/noarch::fonts-conda-forge-1-0 freetype conda-forge/linux-aarch64::freetype-2.12.1-hf0a5ef3_2 gcc conda-forge/linux-aarch64::gcc-13.3.0-h8a56e6e_1 gcc_impl_linux-aa~ conda-forge/linux-aarch64::gcc_impl_linux-aarch64-13.3.0-hcdea9b6_1 gcc_linux-aarch64 conda-forge/linux-aarch64::gcc_linux-aarch64-13.3.0-h1cd514b_5 gds-tools conda-forge/linux-aarch64::gds-tools-1.11.1.6-h3ae8b8a_1 gettext conda-forge/linux-aarch64::gettext-0.22.5-h0a1ffab_3 gettext-tools conda-forge/linux-aarch64::gettext-tools-0.22.5-h0a1ffab_3 gmp conda-forge/linux-aarch64::gmp-6.3.0-h0a1ffab_2 gxx conda-forge/linux-aarch64::gxx-13.3.0-h8a56e6e_1 gxx_impl_linux-aa~ conda-forge/linux-aarch64::gxx_impl_linux-aarch64-13.3.0-h1211b58_1 gxx_linux-aarch64 conda-forge/linux-aarch64::gxx_linux-aarch64-13.3.0-h2864abd_5 icu conda-forge/linux-aarch64::icu-75.1-hf9b3779_0 kernel-headers_li~ conda-forge/noarch::kernel-headers_linux-aarch64-4.18.0-h05a177a_18 keyutils conda-forge/linux-aarch64::keyutils-1.6.1-h4e544f5_0 krb5 conda-forge/linux-aarch64::krb5-1.21.3-h50a48e9_0 ld_impl_linux-aar~ conda-forge/linux-aarch64::ld_impl_linux-aarch64-2.43-h80caac9_2 libasprintf conda-forge/linux-aarch64::libasprintf-0.22.5-h87f4aca_3 libasprintf-devel conda-forge/linux-aarch64::libasprintf-devel-0.22.5-h87f4aca_3 libcap conda-forge/linux-aarch64::libcap-2.69-h883460d_0 libcublas conda-forge/linux-aarch64::libcublas-12.6.3.3-h5101a13_1 libcublas-dev conda-forge/linux-aarch64::libcublas-dev-12.6.3.3-h3ae8b8a_1 libcufft conda-forge/linux-aarch64::libcufft-11.3.0.4-h5101a13_0 libcufft-dev conda-forge/linux-aarch64::libcufft-dev-11.3.0.4-h3ae8b8a_0 libcufile conda-forge/linux-aarch64::libcufile-1.11.1.6-h71dc0f0_1 libcufile-dev conda-forge/linux-aarch64::libcufile-dev-1.11.1.6-h3ae8b8a_1 libcurand conda-forge/linux-aarch64::libcurand-10.3.7.77-h5101a13_0 libcurand-dev conda-forge/linux-aarch64::libcurand-dev-10.3.7.77-h3ae8b8a_0 libcusolver conda-forge/linux-aarch64::libcusolver-11.7.1.2-h5101a13_0 libcusolver-dev conda-forge/linux-aarch64::libcusolver-dev-11.7.1.2-h3ae8b8a_0 libcusparse conda-forge/linux-aarch64::libcusparse-12.5.4.2-h5101a13_0 libcusparse-dev conda-forge/linux-aarch64::libcusparse-dev-12.5.4.2-h3ae8b8a_0 libedit conda-forge/linux-aarch64::libedit-3.1.20191231-he28a2e2_2 libexpat conda-forge/linux-aarch64::libexpat-2.6.3-h5ad3122_0 libffi conda-forge/linux-aarch64::libffi-3.4.2-h3557bc0_5 libgcc conda-forge/linux-aarch64::libgcc-14.2.0-he277a41_1 libgcc-devel_linu~ conda-forge/noarch::libgcc-devel_linux-aarch64-13.3.0-h0c07274_101 libgcc-ng conda-forge/linux-aarch64::libgcc-ng-14.2.0-he9431aa_1 libgcrypt conda-forge/linux-aarch64::libgcrypt-1.11.0-h68df207_1 libgettextpo conda-forge/linux-aarch64::libgettextpo-0.22.5-h0a1ffab_3 libgettextpo-devel conda-forge/linux-aarch64::libgettextpo-devel-0.22.5-h0a1ffab_3 libglib conda-forge/linux-aarch64::libglib-2.82.2-hc486b8e_0 libgomp conda-forge/linux-aarch64::libgomp-14.2.0-he277a41_1 libgpg-error conda-forge/linux-aarch64::libgpg-error-1.50-hb13efb6_0 libiconv conda-forge/linux-aarch64::libiconv-1.17-h31becfc_2 libnl conda-forge/linux-aarch64::libnl-3.10.0-h68df207_0 libnpp conda-forge/linux-aarch64::libnpp-12.3.1.54-h3ae8b8a_0 libnpp-dev conda-forge/linux-aarch64::libnpp-dev-12.3.1.54-h3ae8b8a_0 libnvfatbin conda-forge/linux-aarch64::libnvfatbin-12.6.77-h5101a13_0 libnvfatbin-dev conda-forge/linux-aarch64::libnvfatbin-dev-12.6.77-h3ae8b8a_0 libnvjitlink conda-forge/linux-aarch64::libnvjitlink-12.6.77-h5101a13_1 libnvjitlink-dev conda-forge/linux-aarch64::libnvjitlink-dev-12.6.77-h3ae8b8a_1 libnvjpeg conda-forge/linux-aarch64::libnvjpeg-12.3.3.54-h3ae8b8a_0 libnvjpeg-dev conda-forge/linux-aarch64::libnvjpeg-dev-12.3.3.54-h579c4fd_0 libpng conda-forge/linux-aarch64::libpng-1.6.44-hc4a20ef_0 libsanitizer conda-forge/linux-aarch64::libsanitizer-13.3.0-ha58e236_1 libsqlite conda-forge/linux-aarch64::libsqlite-3.47.0-hc4a20ef_1 libstdcxx conda-forge/linux-aarch64::libstdcxx-14.2.0-h3f4de04_1 libstdcxx-devel_l~ conda-forge/noarch::libstdcxx-devel_linux-aarch64-13.3.0-h0c07274_101 libstdcxx-ng conda-forge/linux-aarch64::libstdcxx-ng-14.2.0-hf1166c9_1 libsystemd0 conda-forge/linux-aarch64::libsystemd0-256.7-hd54d049_1 libudev1 conda-forge/linux-aarch64::libudev1-256.7-h86ecc28_1 libuuid conda-forge/linux-aarch64::libuuid-2.38.1-hb4cce97_0 libxcb conda-forge/linux-aarch64::libxcb-1.17.0-h262b8f6_0 libxkbcommon conda-forge/linux-aarch64::libxkbcommon-1.7.0-h46f2afe_1 libxkbfile conda-forge/linux-aarch64::libxkbfile-1.1.0-h4e544f5_1 libxml2 conda-forge/linux-aarch64::libxml2-2.13.4-hf4efe5d_2 libzlib conda-forge/linux-aarch64::libzlib-1.3.1-h86ecc28_2 lz4-c conda-forge/linux-aarch64::lz4-c-1.9.4-hd600fc2_0 ncurses conda-forge/linux-aarch64::ncurses-6.5-hcccb83c_1 nsight-compute conda-forge/linux-aarch64::nsight-compute-2024.3.2.3-h223ec0a_0 nspr conda-forge/linux-aarch64::nspr-4.36-h5ad3122_0 nss conda-forge/linux-aarch64::nss-3.106-hcffee33_0 openssl conda-forge/linux-aarch64::openssl-3.3.2-h86ecc28_0 pcre2 conda-forge/linux-aarch64::pcre2-10.44-h070dd5b_2 pthread-stubs conda-forge/linux-aarch64::pthread-stubs-0.4-h86ecc28_1002 rdma-core conda-forge/linux-aarch64::rdma-core-54.0-h1d056c8_1 sysroot_linux-aar~ conda-forge/noarch::sysroot_linux-aarch64-2.17-h5b4a56d_18 tzdata conda-forge/noarch::tzdata-2024b-hc8b5060_0 wayland conda-forge/linux-aarch64::wayland-1.23.1-h698ed42_0 xcb-util conda-forge/linux-aarch64::xcb-util-0.4.1-h5c728e9_2 xcb-util-cursor conda-forge/linux-aarch64::xcb-util-cursor-0.1.5-h86ecc28_0 xcb-util-image conda-forge/linux-aarch64::xcb-util-image-0.4.0-h5c728e9_2 xcb-util-keysyms conda-forge/linux-aarch64::xcb-util-keysyms-0.4.1-h5c728e9_0 xcb-util-renderut~ conda-forge/linux-aarch64::xcb-util-renderutil-0.3.10-h5c728e9_0 xcb-util-wm conda-forge/linux-aarch64::xcb-util-wm-0.4.2-h5c728e9_0 xkeyboard-config conda-forge/linux-aarch64::xkeyboard-config-2.43-h86ecc28_0 xorg-libice conda-forge/linux-aarch64::xorg-libice-1.1.1-h57736b2_1 xorg-libsm conda-forge/linux-aarch64::xorg-libsm-1.2.4-hbac51e1_1 xorg-libx11 conda-forge/linux-aarch64::xorg-libx11-1.8.9-he755bbd_2 xorg-libxau conda-forge/linux-aarch64::xorg-libxau-1.0.11-h86ecc28_1 xorg-libxcomposite conda-forge/linux-aarch64::xorg-libxcomposite-0.4.6-h86ecc28_2 xorg-libxdamage conda-forge/linux-aarch64::xorg-libxdamage-1.1.6-h86ecc28_0 xorg-libxdmcp conda-forge/linux-aarch64::xorg-libxdmcp-1.1.5-h57736b2_0 xorg-libxext conda-forge/linux-aarch64::xorg-libxext-1.3.6-h57736b2_0 xorg-libxfixes conda-forge/linux-aarch64::xorg-libxfixes-6.0.1-h57736b2_0 xorg-libxi conda-forge/linux-aarch64::xorg-libxi-1.8.2-h57736b2_0 xorg-libxrandr conda-forge/linux-aarch64::xorg-libxrandr-1.5.4-h86ecc28_0 xorg-libxrender conda-forge/linux-aarch64::xorg-libxrender-0.9.11-h57736b2_1 xorg-libxtst conda-forge/linux-aarch64::xorg-libxtst-1.2.5-h57736b2_3 xorg-xorgproto conda-forge/linux-aarch64::xorg-xorgproto-2024.1-h86ecc28_1 xz conda-forge/linux-aarch64::xz-5.2.6-h9cdd2b7_0 zstd conda-forge/linux-aarch64::zstd-1.5.6-h02f22dd_0 Proceed ([y]/n)? Downloading and Extracting Packages: Preparing transaction: done Verifying transaction: / ClobberWarning: This transaction has incompatible packages due to a shared path. packages: conda-forge/linux-aarch64::cuda-cuxxfilt-12.6.77-h05609ea_1, conda-forge/linux-aarch64::cuda-nvprune-12.6.77-h05609ea_1, conda-forge/linux-aarch64::cuda-sanitizer-api-12.6.77-h4177691_0, conda-forge/linux-aarch64::cuda-cuobjdump-12.6.77-h05609ea_0 path: 'LICENSE' done Executing transaction: | ClobberWarning: Conda was asked to clobber an existing path. source path: /opt/conda/pkgs/cuda-nvprune-12.6.77-h05609ea_1/LICENSE target path: /opt/conda/envs/cuda/LICENSE ClobberWarning: Conda was asked to clobber an existing path. source path: /opt/conda/pkgs/cuda-sanitizer-api-12.6.77-h4177691_0/LICENSE target path: /opt/conda/envs/cuda/LICENSE / ClobberWarning: Conda was asked to clobber an existing path. source path: /opt/conda/pkgs/cuda-cuobjdump-12.6.77-h05609ea_0/LICENSE target path: /opt/conda/envs/cuda/LICENSE done # # To activate this environment, use # # $ conda activate cuda # # To deactivate an active environment, use # # $ conda deactivate ```

The following packages were flagged as having clobbering issues. All with the license file: