This would make an amazing pacman post-transaction hook except..

[Strykar]

I'm unsure how to trigger it only when python/perl is upgraded to a new version or even otherwise TBH (ldd).

FYI, Alad suggested one of:

pactree -slr python | grep -f <(pacman -Slq alad)
pactree -sur python | grep -f <(pacman -Slq alad)

to find aur packages that have Python as dependency but it will also list packages that are not installed. Thanks for this!

[maximbaz]

I'm unsure how to trigger it only when python/perl is upgraded to a new version

Have you tried this?

[Trigger]
Type = Package
Operation = Install
Operation = Upgrade
Target = python
Target = perl

But I guess if you are only interested in python, running the entire checkrebuild might be an overkill, as ldd operation takes a LOT of computational resources that produce no results for languages like python. You might just extract the python check in a separate script and call that one from the hook.

FYI, Alad suggested one of:

Yeah I've seen that, I like the current approach better as it explicitly targets the installed packages 🙂 This is not a tool to manage your local repo (like aurutils), this is a tool to manage your current state of the system.

[Strykar]

Thanks, after discussing on #archlinux and reviewing the Path option, I came up with :

[Trigger]
Operation = Upgrade
Type = Path
Target = /usr/bin/python3 /usr/bin/python2 /usr/bin/perl

[Action]
Description = Perl or Python version upgrade detected, rebuild these packages:
Depends = rebuild-detector
When = PostTransaction
Exec = /usr/bin/checkrebuild

Any chance you'd consider adding a switch to rebuild-detector so just the python or perl checks could be called if desired: checkrebuild -perl -python? This would 1) make it easier to use in a hook and 2) not rely on my bash skills :smile:

If not, in any case, version upgrades for Perl and Python are so few and far apart; the time taken for the ldd function wouldn't matter it'd be called maybe a few times a year, if that.

[maximbaz]

I don't think adding such flags would be a good idea, as the goal of the script is to detect a broken system, not a partially broken one (and also considering how much it would complicate the currently simple script to add flags handling) - it's also quite trivial to take the checkrebuild script and exclude checks that you don't need (i.e. in your case removing line 31).

Nice job figuring out the hook 👍 You can remove python2 target, as checkrebuild is looking at python3 exclusively anyway. Otherwise should work, enjoy!

Out of curiosity, why do you want to run this only for perl/python packages? In my experience, ldd is actually the one that usually discovers troubles, as you said python and perl get updated so rarely it's easy to spot when the rebuild is needed anyway.

[Strykar]

No worries, I'll fork it and make the changes. I also just realized there won't be a python-2.8. I sorta stumbled to wanting a solution when python3 was upgraded and my Powerline scripts for vim stopped working.

My primary reason for using it just for perl and python is due to time spent for the ldd function. I wish to do that check using something written in Rust which is orders of magnitude faster. See check-broken-packages - https://github.com/desbma/pacman-hooks

[maximbaz]

I am not satisfied with the performance of ldd as well, thanks for the link, I'll check it out!

On the one hand, the current script already uses parallel to achieve better performance, so I am a bit skeptical that check-broken-packages can do it faster, but if I manage to get significantly better results I will incorporate them in this project!

[maximbaz]

I had a look, check-broken-packages uses a different approach that is a lot less generic, and very specific to their own setup.

What they do is they first list all packages that pacman considers "foreign", then list all files owned by those packages (all files, not only binaries), and then run ldd on them (in parallel).

The first problem is that not everyone installs AUR packages as foreign, e.g. if you use aurutils, you will have 0 foreign packages on your system, so check-broken-packages will discover nothing.

The second problem is that running ldd on all files owned by a given package is useless, for example chromium-vaapi owns 104 files right now (including images, manual files, etc.), and running ldd on all of them will give you nothing anyway.

My approach in this project is different, I first discover all available files in $PATH (which would mostly consist of binaries that are worth checking with ldd, regardless of how they were installed), and then run ldd only on them (also in parallel).

[Strykar]

Heh, been using Arch for about a year, I have three 'Foreign' packages as I still haven't figured out how to use aurutils with a provided PKGBUILD. Your point stands, most users of aurutils would not.

And running ldd on all files seems wasteful with that approach. Thanks for checking check-broken-packages out in detail, he does mention he's hardcoded stuff for his setup.

Do you suppose it's possible to make the ldd function any faster in shell?

[maximbaz]

Hmm I actually just got an idea how we could try to improve the performance, even two 😄

1. When listing files in $PATH, we could intersect that with the list of files owned by pacman, in other words if you have stuff in $PATH not owned by pacman, running ldd on it will find no packages to rebuild anyway.

2. When listing files in $PATH, we can exclude all files owned by packages in official pacman repos, because those are taken care of by Arch Linux staff anyway.

I will give both ideas a try, I think the first one might prove to be actually slowing down the computation, but the second one might give us something!

[desbma]

The second problem is that running ldd on all files owned by a given package is useless, for example chromium-vaapi owns 104 files right now (including images, manual files, etc.), and running ldd on all of them will give you nothing anyway.

Hi, author of check-broken-packages here. This is an incorrect description of what the program does. ldd is only run on package files with executable bit set.

[maximbaz]

Oh, thanks for the correction, this is nice idea!

[maximbaz]

@Strykar would you like to test the code from latest master? I pushed https://github.com/maximbaz/rebuild-detector/commit/8f1f688af88faf85d07b32a1362dea8acd522b0f (see commit msg for details), the performance is significantly better for me, and also a lot more issues are being discovered now 😄 What are the results for you?

@desbma thanks again for your message and the idea of checking only executable files. If you are interested to generalize your tool, I would recommend changing check-broken-packages to validate all installed packages from non-official repos, not only "foreign" packages that you currently fetch using pacman -Qqm. Here's how I did it so far:

https://github.com/maximbaz/rebuild-detector/blob/8f1f688af88faf85d07b32a1362dea8acd522b0f/checkrebuild#L19-L22

[desbma]

@maximbaz Packages installed from the AUR should be foreign, according to the pacman's definition of that term. If some tools want to mess with that assumption, that is fine but I will not support them.

What you are doing with the above shell code is making a distinction between some repositories, but with Arch's philosophy all packages from available repositories should always form a coherent set (that is why partial upgrade is not supported), so you should never have any broken package in there (unlike with AUR package which are updated without any coherency guarantee). And if you have, there is definitely a problem with theses "unofficial" repositories.

[maximbaz]

There is no such pacman definition that says that AUR packages must be installed as foreign, and for example not being managed via your own unofficial repository 🙂 But it is your choice and I respect it.

@Strykar with the last commit, checkrebuild takes now 1s instead of 8.5s on my own machine, but for me it also finds a lot more results than it used to. While I'm still looking how to fix those packages (if this is even possible), if you have the same experience, you can use checkrebuild -v to print what exactly is wrong.

[maximbaz]

Since it's so fast I added pacman hook as well in 3.0.0, therefore closing 🙂