Open nyanpasu64 opened 2 years ago
nyanpasu64
commented
2 years ago I was writing some code to deduplicate packages pushed into a Base. What should be done if we try to push an AurPackage with the same Package::pkg.name (and probably the same Arc::as_ptr(Package::pkg.0)), but different Package::make and Package::target fields?
Morganamilo
commented
2 years ago Hey this is very detailed thanks. Can't examine it too hard right how as I'm not home.
PARU_DEBUG=1 for debug. Additionally PAUR_ALPM_DEBUG=0 if you're not interested in the alpm side.
Also no hashmaps as there's been some issues with non determinism.
The duplicate package error is not for programmer error. It's for if you have two of the same package in the tree which can happen.
Morganamilo
commented
2 years ago Can you reproduce this with a specific package? Presumably if you -S one of the to be upgraded packages it should cause it.
Morganamilo
commented
2 years ago Also do you use local repos?
nyanpasu64
commented
2 years ago I can't reproduce the bug now.
> cargo run -- --aur -Syu jack-rack --assume-installed hotdoc
Finished dev [unoptimized + debuginfo] target(s) in 0.07s
Running `target/debug/paru --aur -Syu jack-rack --assume-installed hotdoc`
:: Looking for AUR upgrades
:: Looking for devel upgrades
:: Resolving dependencies...
pushing [] jack-rack -> jack-rack
pushing [] python-agate -> python-agate
pushing [] python-crate -> python-crate
pushing [] python-dbfread -> python-dbfread
pushing [] xfce4-docklike-plugin-ng-git -> xfce4-docklike-plugin-ng-git
pushing [] youtubedr -> youtubedr
pushing ["alsa-card-profiles-git"] libcamera-git -> libcamera-git
pushing [] pipewire-git -> alsa-card-profiles-git
pushing [] gdb-git -> gdb-git
pushing ["pipewire-alsa-git", "wireplumber-git"] pipewire-git -> pipewire-git
pushing ["pipewire-alsa-git", "wireplumber-git"] pipewire-git -> pipewire-alsa-git
pushing ["pipewire-alsa-git", "wireplumber-git"] pipewire-git -> pipewire-jack-git
pushing ["pipewire-alsa-git", "wireplumber-git"] pipewire-git -> pipewire-pulse-git
pushing ["pipewire-alsa-git"] wireplumber-git -> wireplumber-git
pushing [] rr-git -> rr-git
pushing [] wlroots-eglstreams-git -> wlroots-eglstreams-git
:: Calculating conflicts...
:: Calculating inner conflicts...I'm not sure why installing jack-rack doesn't directly rebuild pipewire-jack-git. However, uninstalling ocenaudio-bin and running cargo run -- --aur -Su ocenaudio-bin reproduces the bug for me.
Also do you use local repos?
No, what's that?
Morganamilo
commented
2 years ago Local repos are just well local pacman repos.
Anyway is that output from when you couldn't reproduce it? The output of it happening with debug output enabled would be useful.
nyanpasu64
commented
2 years ago Anyway is that output from when you couldn't reproduce it?
Yes.
The output of it happening with debug output enabled would be useful.
bug.txt: PARU_DEBUG=1 PAUR_ALPM_DEBUG=0 cargo run -- --aur -Su ocenaudio-bin 2>bug.txt
no-bug.txt: PARU_DEBUG=1 PAUR_ALPM_DEBUG=0 cargo run -- --aur -Su jack-rack 2>no-bug.txt
Both files contain my pushing messages as well as debug messages.
sukanka
commented
2 years ago I have a similar problem. When I run paru -S r-cli or paru -S r-glue, I get the error message
:: Resolving dependencies...
error: duplicate packages: r-processxIf you could post the full output with PARU_DEBUG=1 that should help track it down.
sukanka
commented
2 years ago Here is the full output. paru_debug.log
RubenKelevra
commented
2 years ago @Morganamilo here's another example:
python-sanic was added to chaos-aur, which lead paru to complain on a simple paru -Syu:
=> paru uses system-config file: /etc/paru.conf
[options]
PgpFetch
Devel
Provides
DevelSuffixes = -git -cvs -svn -bzr -darcs -always -hg -fossil
BottomUp
RemoveMake
SudoLoop
UseAsk
CombinedUpgrade
UpgradeMenu
NewsOnUpgrade
SkipReview
KeepSrc
FailFast
LocalRepo
Chroot = /var/lib/paru/aur_chroot
[bin]
FileManager = nnn[options]
HoldPkg = pacman glibc
Architecture = auto
IgnorePkg = python2 jre libva-intel-driver jre11-openjdk jdk11-openjdk jre8-adoptopenjdk jdk8-adoptopenjdk jre11-openjdk-headless jre8-adoptopenjdk-headless ttf-google-fonts-git
NoExtract = usr/lib/firefox/fonts/TwemojiMozilla.ttf
Color
ILoveCandy
CheckSpace
VerbosePkgLists
DisableDownloadTimeout
ParallelDownloads = 4
SigLevel = Required DatabaseOptional
LocalFileSigLevel = Optional
[core]
Include = /etc/pacman.d/mirrorlist
[extra]
Include = /etc/pacman.d/mirrorlist
[community]
Include = /etc/pacman.d/mirrorlist
[multilib]
Include = /etc/pacman.d/mirrorlist
[endeavouros]
SigLevel = PackageRequired
Include = /etc/pacman.d/endeavouros-mirrorlist
[chaotic-aur]
Include = /etc/pacman.d/chaotic-mirrorlist
[aur-chroot-builds]
SigLevel = PackageOptional DatabaseOptional
Server = file:///var/lib/paru/aur_chroot
[core-debug]
Include = /etc/pacman.d/mirrorlist_debug
[extra-debug]
Include = /etc/pacman.d/mirrorlist_debug
[community-debug]
Include = /etc/pacman.d/mirrorlist_debug
[multilib-debug]
Include = /etc/pacman.d/mirrorlist_debugI can confirm this for paru v1.11.1 - libalpm v13.0.1
Morganamilo
commented
1 year ago I've reproduced this in the past but now I'm trying to fiix it, I can't get it to show up. A lot of the code has changed too so many it got fixed.
pekkarr
commented
1 year ago I got the same error when trying to upgrade r-openssl: error: duplicate packages: r-openssl (this isn't reproducible anymore, see note at the end)
I think this might be related to the fact that there is a semi-cyclical dependency between r-openssl, r-usethis and r-gert:
r-openssl checkdepends on r-usethisr-usethis depends on r-gertr-gert depends on r-opensslWhen building these packages for the first time with check enabled the process should be following:
r-openssl with --nocheckr-gertr-usethisr-openssl (with check)Previously there was also a similar cycle between r-processx, r-ps and r-testthat which would explain why error: duplicate packages: r-processx was reported earlier in this thread. That cycle was removed by r-ps maintainer (me) by removing tests from r-ps which explains why the error can't be reproduced anymore.
I don't know if these semi-cyclical dependencies should be considered as bugs in the PKGBUILD, as it's possible to build them correctly as I stated earlier. However I've been trying to avoid them in my PKGBUILDs because they make building with check enabled a hassle (at least without automated tools).
While I was writing this comment check got removed from r-openssl which removed the dependency cycle. After that change paru was able to successfully build and install the package.
StripedMonkey
commented
1 year ago I seem to be encountering a similar issue at the moment with llvm-git, when updating.
Affected Version
paru -V paru v1.9.2 - libalpm v13.0.1
Description
Have you checked previous issues? yes
Output
Include the FULL output of any relevant commands/configs
Don't cut parts of the input always include the FULL thing
paru.conf and pacman.conf are usually always relevant
paru-conf.zip
Debugging
I spent a day debugging this issue (whew) and eventually traced the problem to
aur-depends. I vendoredaur-dependsand added a print toaur_depends::resolve::Resolver::push_build():I think
pkgbaserefers to the package being built, andpkg.pkg.namerefers to the build artifact which gets installed (thepipewire-gitPKGBUILD generates many binary packages). I thinkResolver::actions.buildis a vector where eachBaseelement holds a vectors of packages with a common PKGBUILD andpkg.package_base(the base package whosepkg.name == pkg.package_basemay be first, not first, or completely absent), excluding packages supplied by the PKGBUILD but not to be installed (eg.pipewire-ffmpeg-gitnever gets pushed tobuild[]).Suggestion: Why not remove the
pkgbaseparameter and usepkg.pkg.package_baseinstead?push_buildgets called exactly twice, and in both casespkg.pkgis a clone of the packagepkgbaseis extracted from.The output is:
Looking at package dependencies:
jackresolves topipewire-jack-gitwhich is installed. Since it's an outdated AUR package, it's pushed to the build list.pipewire-session-managerresolves towireplumber-gitwhich is installed. Since it's an outdated AUR package, it's pushed to the build list.libpipewire-0.3.so=0-64resolves topipewire-gitwhich is installed. Since it's an outdated AUR package, it's pushed to the build list.Why does
wireplumber-gitcallpush_buildonpipewire-jack-gitas well? I think it's becausepipewire-gitalso suppliespipewire-jack-git, which should be upgraded since it's installed but outdated.For some reason,
aur-dependsperforms a postorder traversal and callsresolve_aur_pkg_deps(recursively trace dependencies) beforepush_build(add package to build list). (I'm not sure if this results in a topological sort or not; I've forgotten the algorithms.) Soocenaudio-binpushespipewire-jack-gitafterwireplumber-gitpushespipewire-gitandpipewire-jack-git.At the end,
paru::install::check_actions()complains about duplicate packages being installed:https://github.com/Morganamilo/paru/blob/dbf5d8afb694a87b24318742ab2ef554b07f09bd/src/install.rs#L971-L981
Bug
The problem is that
aur-dependsfails to deduplicate packages being pushed to a givenBaseto be built. I'd either add a check that we don't push duplicate items into aBase(linearly scanningBase::pkgsis fast since theVecis short, since most PKGBUILD don't produce a thousand packages which all get built and installed), or replaceVecwithHashSetorBTreeSetsuch. (Is it a problem if the package order within a PKGBUILD changes from the current algorithm?)If you're interested I have a pernosco debugging trace of this bug (link). It might've helped me learn the code (not sure), but I ended up figuring out the bug by adding a print statement (not by using Pernosco).
Suggestions
Actions::buildfield, to something like: "AUR packages to build. Each Base corresponds to a single PKGBUILD producing one or more packages we want to install." Or clarify the exact wording if it's not 100% accurate.struct Baseto something like "A set of packages to be installed, coming from the same PKGBUILD (and having the same pkg.package_base)." Or clarify the exact wording if it's not 100% accurate.push_build(). I haven't thought of an idea yet.push_build()like myeprintln!. The currentdebug!statement doesn't trip when pushing packages into an existing base, only when you create a new base.resolve_aur_pkg_depsandresolvealready havedebug!logging, but IDK if it's useful or not.debug!logs?paru -S pipewire-git pipewire-git, or runparu -S paru-bin paruand selectparu-bin.Is there architectural documentation?