yay AUR exact match gets buried in list of possible package matches

[ryanerwin]

Affected Version

 ❯ yay --version
yay v9.2.0 - libalpm v11.0.3

Issue

When an AUR package name is an exact match for your search term, the package is frequently still buried in the middle of the list of possible packages. None of the --sortby options will bring your exact match to number one on the results list.

Steps to reproduce

1. First, note that "authenticator" is an exact package name match in AUR https://aur.archlinux.org/packages/authenticator/
2. Run: yay authenticator...
3. Any permutation of yay --sortby <votes|popularity|id|baseid|name|base|submitted|modified> authenticator gets basically the same result.

Output

`yay authenticator`, exact match is #19 in results

``` ❯ yay authenticator 39 aur/perl-auth-googleauthenticator 0.0.3-1 (+0 0.00%) Perl/CPAN module Net::Google::Authenticator 38 aur/gashell 0.91-1 (+0 0.00%) A bash script that generates and securely manages/stores Google Authenticator codes. 37 aur/authenticator-git 3.32.1.r34.g302fb8b-1 (+0 0.00%) 2FA code generator for GNOME 36 aur/certbot-ovh 0.1-1 (+0 0.00%) OVH DNS Authenticator plugin for Certbot 35 aur/openvpn-otp 1.0-3 (+0 0.00%) This plugin adds support for TOTP and HOTP tokens (like Google Authenticator) for OpenVPN. 34 aur/aws-iam-authenticator-git v0.4.0.r5.gc2d2884d-1 (+0 0.00%) A tool to use AWS IAM credentials to authenticate to a Kubernetes cluster 33 aur/authprogs-git a-1 (+0 0.00%) SSH Command Authenticator 32 aur/certbot-dns-inwx-git 20180409.d59cb0f-1 (+0 0.00%) INWX DNS authenticator plugin for certbot - GIT version 31 aur/certbot-dns-henet-git r16.931e2c7-1 (+0 0.00%) he.net DNS Authenticator plugin for Certbot 30 aur/certbot-dns-alwaysdata 0.27.1-1 (+0 0.00%) Alwaysdata DNS Authenticator plugin for Certbot 29 aur/roundcube-twofactor-git r48.a2d33ef-1 (+1 0.00%) (Out-of-date 2019-03-10) Adds two-factor authentication through Google Authenticator. 28 aur/hostapd-rtl 2.4-2 (+2 0.00%) (Orphaned) (Out-of-date 2018-02-03) IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator 27 aur/pwauth 2.3.11-6 (+3 0.00%) An authenticator designed to be used for web authentication 26 aur/keepass-plugin-keeotp-beta 1.3.7-1 (+3 0.00%) (Out-of-date 2017-01-13) KeePass plugin that generates TOTPs from secrets stored in the KeePass database (ie: Google Authenticator, other RFC 6238 compliant TOTP implementations) (Beta version) 25 aur/hostapd-git 20170315.6c2056abe-1 (+3 0.00%) IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator 24 aur/aws-iam-authenticator-bin 0.4.0-1 (+3 0.07%) A tool to use AWS IAM credentials to authenticate to a Kubernetes cluster 23 aur/jauth-git r63.bbe0a11-1 (+4 0.00%) A reference desktop client for the google authenticator 22 aur/python-bna 4.1.0-1 (+4 0.00%) Command-line Battle.net authenticator and Python library 21 aur/steamguard-cli-git 0.3.2.0.r1.g8c11d22-1 (+5 0.00%) A linux utility for setting up and using Steam Mobile Authenticator (AKA Steam 2FA) on the command line 20 aur/hostapd-rtl871xdrv 2.6-2 (+8 0.00%) IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator 19 aur/authenticator 3.32.1-3 (+8 0.84%) 2FA code generator for GNOME 18 aur/keepass-plugin-keeotp 1.3.9-1 (+9 0.00%) KeePass plugin that generates TOTPs from secrets stored in the KeePass database (ie: Google Authenticator, other RFC 6238 compliant TOTP implementations) 17 aur/google-authenticator-libpam-git r120.c1b3879-1 (+32 0.00%) PAM module for google authenticator app 16 community/libpam-google-authenticator 1.06-1 (37.4 KiB 141.0 KiB) PAM module for google authenticator app 15 community/hostapd 2.8-1 (506.7 KiB 1.4 MiB) IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator 14 community/certbot-dns-sakuracloud 0.35.1-1 (8.9 KiB 21.0 KiB) Sakura Cloud DNS Authenticator plugin for Certbot 13 community/certbot-dns-route53 0.35.1-1 (13.3 KiB 41.0 KiB) Route53 DNS Authenticator plugin for Certbot 12 community/certbot-dns-rfc2136 0.35.1-1 (13.3 KiB 43.0 KiB) NS1 DNS Authenticator plugin for Certbot 11 community/certbot-dns-ovh 0.35.1-1 (9.4 KiB 22.0 KiB) OVH DNS Authenticator plugin for Certbot 10 community/certbot-dns-nsone 0.35.1-1 (8.8 KiB 20.0 KiB) NS1 DNS Authenticator plugin for Certbot 9 community/certbot-dns-luadns 0.35.1-1 (8.8 KiB 20.0 KiB) LuaDNS DNS Authenticator plugin for Certbot 8 community/certbot-dns-linode 0.35.1-1 (9.8 KiB 29.0 KiB) Linode DNS Authenticator plugin for Certbot 7 community/certbot-dns-google 0.35.1-1 (22.0 KiB 107.0 KiB) Google Cloud DNS Authenticator plugin for Certbot 6 community/certbot-dns-gehirn 0.35.1-1 (8.9 KiB 21.0 KiB) Gehirn DNS Authenticator plugin for Certbot 5 community/certbot-dns-dnsmadeeasy 0.35.1-1 (9.0 KiB 21.0 KiB) DNS Made Easy DNS Authenticator plugin for Certbot 4 community/certbot-dns-dnsimple 0.35.1-1 (8.7 KiB 19.0 KiB) DNSimple DNS Authenticator plugin for Certbot 3 community/certbot-dns-digitalocean 0.35.1-1 (11.7 KiB 35.0 KiB) DigitalOcean DNS Authenticator plugin for Certbot 2 community/certbot-dns-cloudxns 0.35.1-1 (8.9 KiB 20.0 KiB) CloudXNS DNS Authenticator plugin for Certbot 1 community/certbot-dns-cloudflare 0.35.1-1 (12.2 KiB 39.0 KiB) Cloudflare DNS Authenticator plugin for Certbot ==> Packages to install (eg: 1 2 3, 1-3 or ^4) ==> ```

`yay -Pg`

``` ❯ yay -Pg { "aururl": "https://aur.archlinux.org", "buildDir": "/home/ryan/.cache/yay", "editor": "", "editorflags": "", "makepkgbin": "makepkg", "makepkgconf": "", "pacmanbin": "pacman", "pacmanconf": "/etc/pacman.conf", "tarbin": "bsdtar", "redownload": "no", "rebuild": "no", "answerclean": "", "answerdiff": "", "answeredit": "", "answerupgrade": "", "gitbin": "git", "gpgbin": "gpg", "gpgflags": "", "mflags": "", "sortby": "votes", "gitflags": "", "removemake": "ask", "requestsplitn": 150, "sortmode": 0, "completionrefreshtime": 7, "sudoloop": false, "timeupdate": false, "devel": false, "cleanAfter": false, "gitclone": true, "provides": true, "pgpfetch": true, "upgrademenu": true, "cleanmenu": true, "diffmenu": true, "editmenu": false, "combinedupgrade": false, "useask": false } ```

My preference would be that an exact match between the yay parameter in the package name, would always be 1st on the list, with additional versions of that same package (authenticator-git in this case) should come next, and then the search results after that.

[Jguer]

https://github.com/Jguer/yay/blob/f406866a458d1dd5ed737b9671377002e1c46d5d/query.go#L104

Leaving this here as it is the relevant function doing this.

[Morganamilo]

I have nothing against changing the sorter to prefer exact matches. But if you're looking for an exact match up front then just use -S.

[SampsonCrowley]

I wish I knew go so I could make this happen. really sick of having to search through a giant list to update IgnorePkg items like postgres & postgres-libs

Also how is "postgresql" not the number 1 match to searching "postgresql" anyways?

[SampsonCrowley]

I would really like to know how ordering is handled, since a lot of the time it seems that less exact matches are shown before closer exact matches

[Morganamilo]

Searches are ranked by popularity, or was it votes? One of the two.

[Jguer]

AUR ones, by vote almost sure Repo ones, by repository (if you change the order of your repos in /etc/pacman.conf you'll see your searches move accordingly

[core]
Include = /etc/pacman.d/mirrorlist

[extra]
Include = /etc/pacman.d/mirrorlist

[community]
Include = /etc/pacman.d/mirrorlist

makes it print core->extra->community in search. Inside each repo its' alphabetic sort <- this is where improvements can be made (or not, depends on what you like) or you can make a complete cocktail and mix all packages from all repos ( possibly rating 0% to 100% the match and ordering by that metric)

[Jguer]

For repos this is used to get candidates: https://github.com/Jguer/yay/blob/f406866a458d1dd5ed737b9671377002e1c46d5d/query.go#L261

To print: https://github.com/Jguer/yay/blob/f406866a458d1dd5ed737b9671377002e1c46d5d/print.go#L107

[Fabian42]

I'm using yay v9.4.6 - libalpm v12.0.1 and community/virtualbox is the 31st of 89 search results for "virtualbox".

[Fabian42]

Another highlight: extra/tk is the 62nd out of 1935 search results for "tk". If I zoom out enough in the console to see it, I can barely read anything anymore. The sixth result is this:

6 extra/avahi 0.8+15+ge8a3dd0-1 (435.9 KiB 1.8 MiB) (Installed)
    Service Discovery for Linux using mDNS/DNS-SD -- compatible with Bonjour

Apart from "KiB", this doesn't even contain a "k"!

[grandchild]

Yes, thank you @Fabian42 , I think it is clear what the issue is how especially alphabetically "low" packages from extra and community might be way, way scrolled up the list.

Which of the possible sortings mentioned above you'd prefer might instead be of some interest (rather than simply repeating the already established problem):

1. Exact matches sorted at the front within the repo (e.g. extra/tk sorted first within the extra section), or
2. Exact matches sorted at the global front, (e.g. extra/tk sorted even before core packages)

[SampsonCrowley]

@grandchild I think I'd probably vote 1; though I see merit in either

[Fabian42]

2. There could be a hundred packages in core that don't match at all (like core/linux56 for virtualbox) and only then one in community (like community/virtualbox). So it should even prefer exact matches and partial matches over the repository, only then fuzzy matches in whatever order seems reasonable. Of course there can be options for sorting as well.

[grandchild]

I'm also in favor of option 2.

[karatedog]

Option #2 favored, although it wouldn't save me from getting tons of useless pages if it displayed the exact match in the first line. I searched for fx. Full text search engines like Lucene have long found out how to do proper ordering/ranking in relation of what the user entered and what order the "search" returned hits. Maybe get some ideas from them, obviously the basics would suffice.

Or allow regex to be entered in the search expression. Yay is a technical tool, anyone who uses it will surely have the knowledge at least of the "^" and "$".

[Fabian42]

yay seems to already do something with regex-like searches:

[fabian@manjaro ~]$ yay no[td]epad
2 community/notepadqq 1.4.8-3 (2.9 MiB 10.2 MiB) 
    Notepad++-like text editor for Linux
1 extra/leafpad 0.8.18.1-9 (91.3 KiB 301.0 KiB) 
    A notepad clone for GTK+ 2.0
==> Packages to install (eg: 1 2 3, 1-3 or ^4)
==> 0
 there is nothing to do
[fabian@manjaro ~]$ yay notepad++
6 aur/gamma-text-editor 0.0.4-0 (+0 0.00) 
    A lightweight text editor, alternative to Gedit or Notepad++
5 aur/notepadqq-ote-git 0.52.1.r631.gbfbdf7c-1 (+1 0.00) 
    Notepad++-like editor for the Linux desktop w/ OTE backend
4 aur/notepadqq-legacy 1.3.6-1 (+1 0.00) 
    Notepad++-like text editor for Linux. Legacy version which doesnt ask for restoration at startup.
3 aur/notepadqq-git 2.0.0.beta.r5.g5553c0be-1 (+39 0.00) 
    Notepad++-like text editor for Linux
2 community/notepadqq 1.4.8-3 (2.9 MiB 10.2 MiB) 
    Notepad++-like text editor for Linux
1 extra/leafpad 0.8.18.1-9 (91.3 KiB 301.0 KiB) 
    A notepad clone for GTK+ 2.0
==> Packages to install (eg: 1 2 3, 1-3 or ^4)
==> 0
 there is nothing to do
[fabian@manjaro ~]$ yay No[td]epad
2 community/notepadqq 1.4.8-3 (2.9 MiB 10.2 MiB) 
    Notepad++-like text editor for Linux
1 extra/leafpad 0.8.18.1-9 (91.3 KiB 301.0 KiB) 
    A notepad clone for GTK+ 2.0
==> Packages to install (eg: 1 2 3, 1-3 or ^4)
==> 

No matter how I look at it, I would have expected at least notepadqq-git, notepadqq-legacy and notepadqq-ote-git to appear in the first search, since it includes notepadqq. And the inclusion of leafpad, which only mentions "notepad" in its description, but exclusion of gamma-text-editor, which has "Notepad" in its description, cannot be explained by uppercase/lowercase, because the third search gives the same results as the first. I don't know what exactly is going on here.

yay virtualbox gives 66 results, community/virtualbox being 24th. yay ^virtualbox gives 24 results, which do NOT all start with "virtualbox" or even include it anywhere in name or description and community/virtualbox is 18th. yay community/virtualbox gives 0 results.

It keeps getting weirder the longer I look into it, so I'll stop now.

TLDR: Regex might already be somewhat implemented, but seems broken. That could be another bug report, I guess. Anyway, it's not too related to the ordering issue, because it would only change the selection, not the order within that.

[Jguer]

Version that's currently in yay-git has a new way of sorting results that should hopefully mean better exact matches. The weights still require some tweaking though which is why it's still unreleased