search

ublue-os / main

OCI base images of Fedora with batteries included
https://universal-blue.org/images/main/
Apache License 2.0
508 stars 51 forks source link

feat: use negativo17 in lieu of rpmfusion #636

Closed bsherman closed 1 month ago

bsherman commented 2 months ago

This PR replaces our uses of the rpmfusion package repo with negativo17's fedora-multimedia repo.

The two motivations for this: 1) reduce number of upstream repos on which we depend. (We've already moved to negativo17 for nvidia since the nvidia packaging allowed more choice in which components to install, so we are quite committed in that regard) 2) generally avoid "package skew". Though we have had plenty of skew with rpmfusion, I do not expect that simply changing to a different package provider will solve this. However, there is a chance this PR will help, since much of our "skew" has been around mesa and ffmpeg. This PR changes how we swap out mesa for "less free" versions, and I think it should work more reliably.

I've run builds locally and inspected the changed packages (eg, libheif, ffmpeg, gstreamer1-plugins, pipewire bluetooth, libva and intel drivers, etc) to make sure the new packages provide what was there before.. Instead of some package names being replaced in packages.json they are now "override-replaced" in the install.sh . Also, some package name changes are due to negativo17's conventions or that their packages have more files in a single package vs fedora (eg, libheif from negativo17 has the contents of the both libheif and libheif tools from before).

bsherman commented 2 months ago

As of the initial commit, I've done enough local testing to thing that this reasonably replaces rpmfusion with corresponding packages, except we do lose the ffmpegthumbnailer package which is not in the negativo17 repos. Also, only tested on F40 Silverblue, not other combos.

bsherman commented 1 month ago

If we build this in a backport for F40/F39 in our staging COPR we can move this forward. https://packages.fedoraproject.org/pkgs/ffmpegthumbnailer/ffmpegthumbnailer/

castrojo commented 1 month ago

This issue is blocked by: https://github.com/ublue-os/packages/issues/32

bsherman commented 1 month ago

This issue is blocked by: ublue-os/packages#32

this blocking issue, re: ffmpegthumbnailer, has been closed and the package has been restored to package list in this PR.

bsherman commented 1 month ago

I think this is looking pretty good now. I've run builds locally and inspected the changed packages (eg, libheif, ffmpeg, pipewire bluetooth, libva and intel drivers, etc) to make sure the new packages provide what was there before... instead of some things being added/swapped in packages.json they are now "override-replaced" in the install.sh . and some of the negativo17 packages have more in a single package vs fedora (eg, libheif from negativo17 has the contents of the both libheif and libheif tools from before).

fiftydinar commented 1 month ago

You have libva-intel-media-driver specified in both replace & packages.json. Just visual duplicate, it's harmless. Everything else looks good to me.

bsherman commented 1 month ago

TL;DR the gstreamer-plugin change should have been from this PR. This comment captures the changes so it can be well understood.

Longer version... I may have been a bit eager in swapping the gstreamer plugins from free (fedora) to "full" (negativo17).

It resulted in larger package diffs than I had expected, so I'm documenting them here for reference.

I built custom images from the following build configs:

Changes when rebasing from sb-main-40 rpmfusion to sb-main-40 negativo17:

Upgraded:
  ffmpeg 6.1.2-2.fc40 -> 1:6.1.2-2.fc40
  ffmpeg-libs 6.1.2-2.fc40 -> 1:6.1.2-2.fc40
  heif-pixbuf-loader 1.17.6-1.fc40 -> 1:1.18.2-3.fc40
  libavdevice 6.1.2-2.fc40 -> 1:6.1.2-2.fc40
  libheif 1.17.6-1.fc40 -> 1:1.18.2-3.fc40
  libva 2.21.0-3.fc40 -> 1:2.22.0-1.fc40
  libva-intel-media-driver 24.2.5-1.fc40 -> 1:24.3.4-1.fc40
  libva-utils 2.21.0-1.fc40 -> 1:2.21.0-1.fc40
  libvdpau 1.5-6.fc40 -> 1.5-11.fc40
  mesa-dri-drivers 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-filesystem 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libEGL 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libGL 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libgbm 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libglapi 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-libxatracker 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  mesa-vulkan-drivers 24.1.7-1.fc40 -> 1:24.1.6-1.fc40
  x264-libs 0.164-13.20231001git31e19f92.fc40 -> 1:0.164-34.20240513git4613ac3.fc40
  x265-libs 3.6-2.fc40 -> 1:3.6-10.fc40
Downgraded:
  ffmpegthumbnailer 2.2.3-0.7.20240104git1b5a779.fc40 -> 2.2.2-1.20240105git1b5a779.fc40
  libde265 1.0.15-2.fc40 -> 1.0.15-1.fc40
  libfreeaptx 0.1.1-6.fc40 -> 0.1.1-1.fc40
Removed:
  fdk-aac-free-2.0.0-13.fc40.x86_64
  intel-media-driver-24.2.5-1.fc40.x86_64
  libheif-freeworld-1.17.6-1.fc40.x86_64
  libheif-tools-1.17.6-1.fc40.x86_64
  libva-intel-driver-2.4.1-14.20221130gitab755cb.fc40.x86_64
  mesa-va-drivers-freeworld-24.1.7-1.fc40.x86_64
  pipewire-codec-aptx-1.0.7-1.fc40.x86_64
  rpmfusion-free-release-40-1.noarch
  rpmfusion-nonfree-release-40-1.noarch
  vmaf-models-2.3.0-7.fc40.noarch
Added:
  aribb24-1.0.3^20160216git5e9be27-1.fc40.x86_64
  davs2-libs-1.6-5.20220903gitb41cf11.fc40.x86_64
  ffmpegthumbnailer-libs-2.2.2-1.20240105git1b5a779.fc40.x86_64
  intel-vaapi-driver-1:2.4.1-7.fc40.x86_64
  kvazaar-libs-2.3.1-1.fc40.x86_64
  libaribcaption-1.1.1-1.fc40.x86_64
  libavcodec-1:6.1.2-2.fc40.x86_64
  libavfilter-1:6.1.2-2.fc40.x86_64
  libavformat-1:6.1.2-2.fc40.x86_64
  libavutil-1:6.1.2-2.fc40.x86_64
  libfdk-aac-1:2.0.3-2.fc40.x86_64
  libklvanc-1.6.0-1.fc40.x86_64
  libnpp-1:12.3.1.54-1.fc40.x86_64
  libpostproc-1:6.1.2-2.fc40.x86_64
  libswresample-1:6.1.2-2.fc40.x86_64
  libswscale-1:6.1.2-2.fc40.x86_64
  libxavs-0.1.55-1.fc40.x86_64
  mesa-va-drivers-1:24.1.6-1.fc40.x86_64
  pipewire-libs-extra-1.0.8-1.fc40.x86_64
  svt-hevc-libs-1.5.1-3.fc40.x86_64
  svt-vp9-libs-0.3.0-11.fc40.x86_64
  uavs3d-libs-1.2.0-4.20230223git1fd0491.fc40.x86_64
  uvg266-libs-0.8.1-1.fc40.x86_64
  vvdec-libs-2.3.0-1.fc40.x86_64
  vvenc-libs-1.12.0-1.fc40.x86_64
  xavs2-libs-1.3-5.20190422giteae1e8b.fc40.x86_64

Changes when rebasing from sb-main-40 negativo17 to sb-main-40 negativo17-gstreamer:

Removed:
  gstreamer1-plugins-bad-free-1.24.8-1.fc40.x86_64
  gstreamer1-plugins-bad-free-libs-1.24.8-1.fc40.x86_64
  gstreamer1-plugins-ugly-free-1.24.8-1.fc40.x86_64
Added:
  CharLS-2.4.2-5.fc40.x86_64
  SuperLU-6.0.1-3.fc40.x86_64
  armadillo-12.8.1-1.fc40.x86_64
  arpack-3.9.1-3.fc40.x86_64
  blosc-1.21.6-1.fc40.x86_64
  boost-program-options-1.83.0-5.fc40.x86_64
  cfitsio-4.4.0-2.fc40.x86_64
  flexiblas-3.4.4-1.fc40.x86_64
  flexiblas-netlib-3.4.4-1.fc40.x86_64
  flexiblas-netlib64-3.4.4-1.fc40.x86_64
  flexiblas-openblas-openmp-3.4.4-1.fc40.x86_64
  flexiblas-openblas-openmp64-3.4.4-1.fc40.x86_64
  flexiblas-openblas-serial-3.4.4-1.fc40.x86_64
  fluid-soundfont-common-3.1-32.fc40.noarch
  fluid-soundfont-lite-patches-3.1-32.fc40.noarch
  freexl-2.0.0-7.fc40.x86_64
  gdal-libs-3.8.5-2.fc40.x86_64
  gdcm-3.0.24-2.fc40.x86_64
  geos-3.12.2-1.fc40.x86_64
  gstreamer1-plugins-bad-1:1.24.8-2.fc40.x86_64
  gstreamer1-plugins-ugly-1:1.24.8-1.fc40.x86_64
  hdf-libs-4.2.16.2-1.fc40.x86_64
  hdf5-1.12.1-15.fc40.x86_64
  libaec-1.1.2-1.fc40.x86_64
  libarrow-15.0.2-5.fc40.x86_64
  libarrow-doc-15.0.2-5.fc40.noarch
  libavtp-0.2.0-7.fc40.x86_64
  libdca-0.0.7-11.fc40.x86_64
  libdeflate-1.21-2.fc40.x86_64
  libgeotiff-1.7.1-12.fc40.x86_64
  libgfortran-14.2.1-3.fc40.x86_64
  libgta-1.2.1-12.fc40.x86_64
  libkml-1.3.0-47.fc40.x86_64
  liblrdf-0.5.0-29.fc40.x86_64
  libltc-1.3.2-5.fc40.x86_64
  libmicrodns-0.2.0-9.fc40.x86_64
  liborc1-1.9.4-1.fc40.x86_64
  libpq-16.4-1.fc40.x86_64
  libqhull_r-1:8.0.2-4.fc40.x86_64
  libquadmath-14.2.1-3.fc40.x86_64
  librtmp-1:2.6-1.20240301git6f6bb13.fc40.x86_64
  librttopo-1.1.0-14.fc40.x86_64
  libspatialite-5.1.0-5.fc40.x86_64
  libvo-aacenc-0.1.3-2.fc40.x86_64
  mariadb-connector-c-3.3.8-3.fc40.x86_64
  mariadb-connector-c-config-3.3.8-3.fc40.noarch
  metis-5.1.0.3-5.fc40.x86_64
  minizip-ng-compat-3.0.10-7.fc40.x86_64
  mjpegtools-libs-2.2.1-1.fc40.x86_64
  neon-0.33.0-1.fc40.x86_64
  netcdf-4.9.2-5.fc40.x86_64
  ogdi-4.1.1-1.fc40.x86_64
  openblas-0.3.26-4.fc40.x86_64
  openblas-openmp-0.3.26-4.fc40.x86_64
  openblas-openmp64-0.3.26-4.fc40.x86_64
  openblas-serial-0.3.26-4.fc40.x86_64
  opencv-bgsegm-4.9.0-3.fc40.x86_64
  opencv-calib3d-4.9.0-3.fc40.x86_64
  opencv-core-4.9.0-3.fc40.x86_64
  opencv-dnn-4.9.0-3.fc40.x86_64
  opencv-features2d-4.9.0-3.fc40.x86_64
  opencv-flann-4.9.0-3.fc40.x86_64
  opencv-imgcodecs-4.9.0-3.fc40.x86_64
  opencv-imgproc-4.9.0-3.fc40.x86_64
  opencv-objdetect-4.9.0-3.fc40.x86_64
  opencv-tracking-4.9.0-3.fc40.x86_64
  opencv-video-4.9.0-3.fc40.x86_64
  pakchois-0.4-30.fc40.x86_64
  parquet-libs-15.0.2-5.fc40.x86_64
  proj-9.3.1-4.fc40.x86_64
  proj-data-9.3.1-4.fc40.noarch
  proj-data-ar-9.3.1-4.fc40.noarch
  proj-data-at-9.3.1-4.fc40.noarch
  proj-data-au-9.3.1-4.fc40.noarch
  proj-data-be-9.3.1-4.fc40.noarch
  proj-data-br-9.3.1-4.fc40.noarch
  proj-data-ca-9.3.1-4.fc40.noarch
  proj-data-ch-9.3.1-4.fc40.noarch
  proj-data-de-9.3.1-4.fc40.noarch
  proj-data-dk-9.3.1-4.fc40.noarch
  proj-data-es-9.3.1-4.fc40.noarch
  proj-data-eur-9.3.1-4.fc40.noarch
  proj-data-fi-9.3.1-4.fc40.noarch
  proj-data-fo-9.3.1-4.fc40.noarch
  proj-data-fr-9.3.1-4.fc40.noarch
  proj-data-is-9.3.1-4.fc40.noarch
  proj-data-jp-9.3.1-4.fc40.noarch
  proj-data-mx-9.3.1-4.fc40.noarch
  proj-data-nc-9.3.1-4.fc40.noarch
  proj-data-nl-9.3.1-4.fc40.noarch
  proj-data-no-9.3.1-4.fc40.noarch
  proj-data-nz-9.3.1-4.fc40.noarch
  proj-data-pl-9.3.1-4.fc40.noarch
  proj-data-pt-9.3.1-4.fc40.noarch
  proj-data-se-9.3.1-4.fc40.noarch
  proj-data-si-9.3.1-4.fc40.noarch
  proj-data-sk-9.3.1-4.fc40.noarch
  proj-data-uk-9.3.1-4.fc40.noarch
  proj-data-us-9.3.1-4.fc40.noarch
  proj-data-za-9.3.1-4.fc40.noarch
  raptor2-2.0.15-43.fc40.x86_64
  re2-1:20220601-19.fc40.x86_64
  spandsp-0.0.6-18.fc40.x86_64
  tbb-2021.11.0-5.fc40.x86_64
  thrift-0.15.0-4.fc40.x86_64
  unixODBC-2.3.12-4.fc40.x86_64
  utf8proc-2.7.0-7.fc40.x86_64
  wildmidi-libs-0.4.5-4.fc40.x86_64
  xerces-c-3.2.5-2.fc40.x86_64
  zbar-libs-0.23.93-3.fc40.x86_64
  zxing-cpp-2.0.0-6.fc40.x86_64
m2Giles commented 1 month ago

That looks like a ton of linear algebra libraries.