search

fosskers / aura

A multilingual package manager for Arch Linux and the AUR.
https://fosskers.github.io/aura/
GNU General Public License v3.0
1.76k stars 114 forks source link

[Enhancement] Native search that spans both repos and AUR (again) #644

Closed ratijas closed 4 years ago

ratijas commented 4 years ago

Summary

Aura needs a native support, e.g. special combination of flags, to search both repos and AUR in one shot.

Description

Native support for a 'wide' search has been an issue for a long time. It has been discussed in 2013 (#88), but I am not happy with the resolution. Since then, the officially suggested way to search both locations is no define a shell function which would call aura twice, passing the same search term to two different modes of operation, namely -As and -Ss.

I don't like the proposed solution, because

  1. it leaves the actual problem unsolved, and
  2. it leverages implementation onto end-users — every single one of them has to edit their .${SHELL}rc ... and all that only because you guys didn't come up with a reasonable combination of flags.

Besides, as noted in #639 ,aura fails with a flashy error message and a non-zero status code whenever it cannot find search matches in the repositories.

Also, the line drawn between repos and AUR operations in aura is unnecessary thick. Like, if I ever have to do either one task or another, but never both at the same time. It may be the right way conceptually, but for a regular user it introduces additional complexity. Honestly, I think the main reason here is the possible complications in code logic, and that it was decided to lift that complexity onto end-users. The general question of this paragraph is: Why AUR packages are considered so second-grady?

Alternatives

Pacaur searches everywhere with -Ss options, and installs from anywhere with -S option. Very convenient, recommend, 5/5.

Yay "extends pacman operations": 

       -S, -Si, -Sl, -Ss, -Su, -Sc, -Qu
              These operations are extended to support both AUR and repo packages.

Version

$ aura -Qi aura
Name            : aura-bin
Version         : 3.1.9-1
Build Date      : Tue Sep 29 16:46:45 2020
fosskers commented 4 years ago

I can add a --wide (or similarly named) option to -A that would do as you want, but in general the whole point of Aura is that -S and -A are separated. AUR packages are second-class because that's the way the Arch world itself treats them. It also makes it clear that "Pacman is Pacman and Aura is Aura". Knowing ahead of time that everything is separate lets people worry less about what Aura might be doing "behind the scenes" when a traditional Pacman option (like -S) is used.

fosskers commented 4 years ago

Thank you for all these issue reports by the way, they are a great help.

fosskers commented 4 years ago

I have an initial form of it working.

stef204 commented 4 years ago

I can add a --wide (or similarly named) option to -A that would do as you want, but in general the whole point of Aura is that -S and -A are separated. AUR packages are second-class because that's the way the Arch world itself treats them. It also makes it clear that "Pacman is Pacman and Aura is Aura". Knowing ahead of time that everything is separate lets people worry less about what Aura might be doing "behind the scenes" when a traditional Pacman option (like -S) is used.

For whatever it's worth, as a long-term user of aura, I completely agree with your response, and with due respect to other users, I don't think this feature is really desirable. Closing some of the existing issues like the handling of Git packages, etc., would seem to be more urgent and valuable. (cf: Arch's own treatment of official Vs. AUR packages.)

fosskers commented 4 years ago

Thanks for your ongoing support :)

As for wide search, as a subflag it was fairly cheap to add. If people need it for scripting purposes or otherwise, it's there. Today I'll be tackling the Git issues.

ratijas commented 4 years ago

Closing some of the existing issues like the handling of Git packages, etc., would seem to be more urgent and valuable.

First of all, I'm sorry, but dealing with other issues has nothing to do with this topic.

Arch's own treatment of official Vs. AUR packages

Arch's own treatment is that AUR packages should be built from scratch with bare hands via makepkg and friends. People found that inconvenient, hence why AUR helpers came into existence in the first place.

If people need it for scripting purposes or otherwise, it's there.

I'm afraid, you got it wrong. I don't need any pacman/aura scripting. It's only for my daily administration. Look at my typical scenario:

I just heard about some software called, say "Deadbeef", I want to try it, I don't want to browse the web looking for package information, I have no business getting into nasty details of packaging. So I just type in pareps deadbeef (which is oh-my-zsh alias for pacaur -Ss) — and there I see all of my options, and there are origin prefixes that allow me to clearly see where those packages come from: aur/deadbeef or community/deadbeef etc. Now my only next step, regardless of the origin, is the command pain !$ which is an ultimate shortcut for pacaur -S. Simple & EZ.

Oh, and here my ultimate argument punch line: you won't have separate flag for removing AUR packages, wouldn't you?

fosskers commented 4 years ago

-S and -A deal with the sources of the packages. Once they're installed, they're installed. So it makes sense that commands like -Q and -R are used for both, regardless of where they came from.

If you already have an alias for the search, then you're free to alias the new subflag as well. The output with the new subflag is exactly as you're describing: 

> aura -Ars snes --head=5
aur/snes9x-git 1.60.r280.gfc78065-1 (15 | 0.26)
    portable Emulator for the Super Nintendo Entertainment System
aur/snes9x-gtk-git 1.60.r280.gfc78065-1 (15 | 0.26)
    Portable Emulator for the Super Nintendo Entertainment System - GTK version
aur/cmdpack-brrrip 1.06-2 (13 | 0.00)
    Rip SNES BRR sound samples
aur/autozen 2.1-3 (12 | 0.00)
    A binural brainwave generator. Cause the user to experience an altered state of consciousness. Wear headphones. Don't use if you suffer for any kind of epilepsy and use at your own risk!
aur/bsnes 115-2 (10 | 2.16)
    Super Nintendo emulator focusing on performance, features, and ease of use.
community/kodi-addon-game-libretro-snes9x 1.60.0.14-4 (kodi-addons kodi-addons-game)
    Libretro wrapper for Kodi's Game API
community/libretro-bsnes 1:1201-1 (libretro)
    Super Nintendo Entertainment System cores
community/libretro-bsnes-hd 23-1 (libretro)
    Super Nintendo Entertainment System core
community/libretro-bsnes2014 1:566-1 (libretro)
    Super Nintendo Entertainment System cores
community/libretro-snes9x 1792-1 (libretro)
    Super Nintendo Entertainment System core
community/snes9x 1.60-6 [installed]
    Portable Super Nintendo Entertainment System (TM) emulator
community/snes9x-gtk 1.60-6 [installed]
    Portable Super Nintendo Entertainment System (TM) emulator
multilib/zsnes 1.51-20
    Super Nintendo emulator
ratijas commented 4 years ago

The output with the new subflag is exactly as you're describing:

Are they sorted by name including prefix? I found it odd to showing AUR packages in front of [official] repositories; especially since you case so much about Arch's treatment. I know I may sound as devil's advocate in this discussion, but if anything, whenever there is a pre-made binary package available from official repos — I would definitely give it a priority.

Otherwise, that sounds reasonable and might work out just good enough. Idk why you chose -r and kind of mnemonic it would have. Suggested earlier -w, --wide is not taken by anything.

fosskers commented 4 years ago

-Aw is actually taken, it downloads a tarball of the package. I'm all-ears for a better option name than -r though. That seemed the obvious one, standing for "repo". Although yes, it's silly that it disagrees with both. Could be b, too.

ratijas commented 4 years ago

-Aw is not present in aura's man page. In fact, there are no -w whatsoever. Need to fix that too, I suppose.

At this rate, the most reliable way to figure out all free letters is to check the source code itself.

fosskers commented 4 years ago

I keep the manpage up to date, I think -w was just an oversight. There was a time where I had removed it as a feature, but added it back again.

As for the crappy --help output (which I'm quite embarrassed of), see #657 .

ratijas commented 4 years ago

Not sure whether it's been reported already, here's one more case of confusing inconsistency:

% auras () {
        aura -Ss $* && aura -As $*
}
% auras qt quick 3d

extra/qt5-quick3d 5.15.1-1 (qt qt5)
    Qt module and API for defining 3D content in Qt Quick
Invalid argument `quick'

Usage: aura ((-A|--aursync) PACKAGES | (-B|--save) | (-C|--downgrade) 
              ([-u|--uninstalled] | PACKAGES) |
              (-L|--viewlog) | (-O|--orphans) | (-P|--analysis) | 
              (-V|--version) | --languages | --viewconf | 
              ((-D|--database) PACKAGES | (-F|--files) | (-Q|--query) 
                [[PACKAGES]] |
                (-R|--remove) PACKAGES | (-S|--sync) [PACKAGES] | (-T|--deptest)
                PACKAGES |
                (-U|--upgrade) PACKAGES)) [--log-level debug|info|warn|error]

The first search result was correctly narrowed down by all three search terms. Second search blew up with an error.

Note that the function auras is a modified version of what was suggested in man page, although man page only passes $1 instead of $*:

function search() {
    aura -Ss $1 && aura -As $1
}
fosskers commented 4 years ago

The AUR search endpoint doesn't quite work the same way, unfortunately. While a search for qt quick 3d on the AUR webpage yields a nice honed-in list, I can't seem to replicate that via the JSON interface.

I think I have a way around that issue, though. Trying it now.

ratijas commented 4 years ago

Oh, that sucks :(

Search page is prerendred, so I guess there's not much we could do — other than manually computing intersection of individual search results. Which is horrible - but not so bad - idea after all, better than showing Usage output anyway. Who's to say that aura must stick only to what JSON API provides us with?

fosskers commented 4 years ago

Exactly, and that's what I've done:

> aura -As qt quick 3d
aur/mingw-w64-qt6-quick3d 6.0.0beta1-1 (0 | 0.00)
    Qt module and API for defining 3D content in Qt Quick (mingw-w64)
aur/qt6-quick3d 6.0.0beta1-1 (0 | 0.00)
    Qt module and API for defining 3D content in Qt Quick
aur/qt5-quick3d-git r837.10f50fb-1 (0 | 0.00)
    a high level 3D API for Qt Quick to introduce 3D content into Qt Quick applications
aur/qt5-canvas3d 5.12.4-1 (3 | 0.36)
    A JavaScript 3D rendering API for Qt Quick (deprecated in qt 5.13)
aur/mingw-w64-qt5-canvas3d 5.12.3-1 (0 | 0.00)
    A JavaScript 3D rendering API for Qt Quick (mingw-w64)
fosskers commented 4 years ago

And the final result:

> aura -Ars qt quick 3d --head=5
aur/qt5-canvas3d 5.12.4-1 (3 | 0.36)
    A JavaScript 3D rendering API for Qt Quick (deprecated in qt 5.13)
aur/mingw-w64-qt6-quick3d 6.0.0beta1-1 (0 | 0.00)
    Qt module and API for defining 3D content in Qt Quick (mingw-w64)
aur/qt6-quick3d 6.0.0beta1-1 (0 | 0.00)
    Qt module and API for defining 3D content in Qt Quick
aur/qt5-quick3d-git r837.10f50fb-1 (0 | 0.00)
    a high level 3D API for Qt Quick to introduce 3D content into Qt Quick applications
aur/mingw-w64-qt5-canvas3d 5.12.3-1 (0 | 0.00)
    A JavaScript 3D rendering API for Qt Quick (mingw-w64)
extra/qt5-quick3d 5.15.1-1 (qt qt5)
    Qt module and API for defining 3D content in Qt Quick
ratijas commented 4 years ago

Now, I've just updated aura, and the issue is its --help output. It contains both these lines slightly separated by dozens of other options:

  -r,--both                With -s: Search the official repos as well.
  -r,--root PATH           Use an alternate installation root.

Autocompletion in zsh also suggests only second variant at the end of aura -A. (I know, I know, I'll fix zsh completions soon. I'm still working on another stuff.)

ratijas commented 4 years ago

aura -Aury is almost like aura -Aura... :thinking:

Would be co cool to have that as an ultimate system upgrade command.

ratijas commented 4 years ago

Ugh, snap it!

aura -Aury / aura -Auy --both does not attempt to perform system upgrade from repositories — only from AUR.

Still not legit substitute for pacaur -Suy. Reopen the issue, please. :slightly_smiling_face:

fosskers commented 4 years ago

-r won't be expanded to -u. It's part of the Aura philosophy that Pacman and AUR operations are separate. Adding -r as a subflag to -s was a one-time exception.

ratijas commented 4 years ago

I'm kinda not much into philosophy, I'm more on the "get things done" side. Guess I still can do a fork, which would be a nightmare to maintain.

fosskers commented 4 years ago

I understand and appreciate your concern here. It's actually that same motivation that made me "draw the line" all the years ago about what Aura is and isn't. From Aura's perspective, it's a feature, not a bug, that -Su is physically/emotionally separate from -Au. I personally don't use pacman at all, only aura, and most of the time when I -Su, I don't want to rebuild a bunch of AUR packages. I -Au when I'm good and ready :)

Perhaps this doesn't match your personal workflow, and so I see how the separation would feel like an inconvenience. "Why do I have to do two commands to get one thing done?" And once again it's Aura philosophy, it's a "ground rule", that updating repo packages and updating AUR packages are in fact two very different things. The Arch folks don't approve of AUR-compatible package managers because they make doing a "dangerous thing" (i.e. building AUR packages) too easy. And by having -S manage AUR packages as well, yes, that is too easy in my view. That's why Aura strides the middle line, offering -A, so that by using it you are conscious of and agree to the risks of building AUR packages.