Arch: install from AUR

[cmosher01]

It would be great to have salt install packages from the AUR for Arch:

[root@archiso salt]# cat /srv/salt/top.sls 
base:
  '*':
    - xa
[root@archiso salt]# cat /srv/salt/xa.sls 
xa:
  pkg:
    - installed
[root@archiso salt]# salt-call -l debug state.highstate --local
[DEBUG   ] Reading configuration from /etc/salt/minion
...
[ERROR   ] The following packages failed to install/update: xa.
...

[techhat]

@cmosher01 I concur. @terminalmage and @cedwards, what would be the most appropriate package managers to use with this, and the most appropriate way to use them?

[terminalmage]

As with anything, people have their preferences, so we'd either need to support more than one AUR frontend or stick with one and deal with the inevitable feature requests asking for their favorite frontend to be supported.

The main ones I hear mentioned are yaourt, cower, and packer.

[cmosher01]

I didn't mean supporting a front end for AUR. I meant having salt install from the AUR instead of from the Official Repositories. Which https://wiki.archlinux.org/index.php/AUR_Helpers , if any, salt uses under the covers shouldn't really matter to the end user.

[techhat]

Oh. Is this missing something that you need?

https://aur.archlinux.org/packages/salt/

[terminalmage]

Are you talking about having salt fetch the tarball and run makepkg? If so, what about dependencies? I think that there is an expectation in salt that the pkg modules install and pull in deps as necessary. This means that, if we're not using an AUR helper, we have to resolve dependencies ourselves. This is not something that I am at all interested in doing.

[cmosher01]

@techhat No. Not relevant. @terminalmage Yes, exactly. And, yes, with dependencies, I agree. If you did not want to do it yourself, then possibly this helper would work: https://wiki.archlinux.org/index.php/Pacaur

[cmosher01]

It looks like the logic could be as simple as this: If pacaur is installed, use "pacaur", else use "pacman", for package commands.

[bsdlp]

actual support for installing packages from AUR instead of shelling out to a helper would be greatly appreciated.

[terminalmage]

That is probably not going to happen, as I noted in my earlier comment. Writing something to resolve dependencies would be tedious and error-prone.

[gtmanfred]

Here is the problem with this whole thing. The only way to truely resolve packages in the AUR is to source the PKGBUILD. There are too many edge cases. and doing --noconfirm on packages that are written by people who do them wrongly and the posibility of someone putting malicious code into PKGBUILDs. There is not a way to trust and verify packages in the AUR yet. Maybe one day if there is a key based system so that PKGBUILDs are signed by gpg keys, and certain users can be trusted packagers in the AUR, then this might work, but until then I don't think it is something that should be done.

The best option is to just make your own private repository

https://wiki.archlinux.org/index.php/Pacman_tips#Custom_local_repository https://wiki.archlinux.org/index.php/Local_repository

[terminalmage]

:+1:

[cmosher01]

Doesn't pacaur handle all the dependency stuff? Seems really simple to me.

[gtmanfred]

pacaur just uses cower to get the dependencies. And cower just tries to parse for the depends() and makedepends(). But you are still blindly running bash scripts that someone who you have no way of verifying has written.

That is not safe, and shouldn't be included in my opinion, until there is a list of verified and trusted aur packages.

And even then, there is a large set of pkgbuilds that might not work, especially with the inclusion of split packages now, where you want to put the different dependencies of each package in the package() function. None of these dependencies will get detected. So, if anything, it shouldn't do any dependency checking, and you should do it manually from pkg states.

I would be ok with just having a thing that downloads the one pkgbuild, and runs makepkg on it. I think that would be fine. There would be no automated dependency checking though. It is still bad, but it is slightly less bad, because you will have at least read the PKGBUILD once.

[gtmanfred]

If anything, maybe https://github.com/KaiSforza/pywer or https://github.com/Kwpolska/pkgbuilder could be used. But even then...

[cmosher01]

@gtmanfred: There may be some cases where a unix admin has no idea what is in the packages he is installing. That may be true. However, if she were in fact a responsible unix admin, and was absolutely sure of the AUR packages she wanted to install, it would be so easy for her to use Salt to install them, if only it would provide (at least a non-defaulted optional) way to install from the AUR. Otherwise it's just not worth using Salt for Arch.

[gtmanfred]

I completely and 100% disagree with that. IIRC, salt was originally run on Arch Linux production server. The correct way to package aur packages is in a custom repository that you drop into /etc/pacman.conf

Now, beyond that. I would be willing to write part of this, but only up to the point of pulling down one package and building it, provide the name and it downlaods from the aur the tarball, or give a link to a build tarball and it will run makepkg on it.

[cmosher01]

You completely disagree that a responsible unix admin would want to install from the AUR using Salt? Really? It would actually be much easier to implement, and more helpful to the end user, to just use pacaur instead of rolling your own.

[gtmanfred]

Why would we use pacaur, when all the dependency checking you want is already done by cower.

And I completely disagree that you need to install things from the aur. It would be responsible to package them locally, test them locally, then distribute them to your servers after testing in your own environment, instead of blindly trusting that other randoms out on the internet know what they are actually doing, and just building their packages and installing them.

The actual repo packages are tracked and signed off on, and reported on, and have a bug tracker for. The aur is just a forum with tarballs and very little accountability.

[cmosher01]

So the driving force behind your objection is that the AUR is only for irresponsible Arch admins. And Salt is only for responsible admins. My point of view goes more along with the traditional unix view of giving admins enough rope to hang themselves with. I want to set up a quick temporary VM of Arch with some AUR packages. Not production servers or anything, just VMs for some testing (... maybe I'm testing AUR packages to see if they actually are safe). I would love be able to just use salt to install them, instead of "correctly" setting up a custom local repo and all that. If the AUR package really doesn't work, then I would have wasted all that time setting up a local repo, etc., to do it "correctly."

[techhat]

By the time an issue moves from discussion to argument, it has gone too far. Plenty of people (including myself) have been using Salt on Arch in production for years, so that is not the issue here. It is clear that @cmosher01 has a use case for which the underlying functionality has not been implemented, and this issue will remain open until a developer has the time and resources to make it happen. When such a time happens, I expect said developer will ask @cmosher01 for additional information if necessary.

Thanks to all of those who have provided insight so far. I expect it will all be taken into consideration when that time comes.

[cmosher01]

Fair enough. I'll be available if and when the time comes.

[rmarquis]

Here is the problem with this whole thing. The only way to truely resolve packages in the AUR is to source the PKGBUILD.

That's not true. Since AUR 3.0.0 has been released a few weeks ago, it is possible to use the extended AUR API for complete dependency information. Unlike PKGBUILD parsing, this method is secure and efficient. That is what should be done here, and focusing on any particular helper is irrelevant imho.

(note: I'm pacaur's maintainer. The next version will handle dependencies through the extended AUR API).

[terminalmage]

@Spyhawk Doesn't this require that the source tarball is built with mkaurball, so that there is an .AURINFO file?

[rmarquis]

Yes. Newly uploaded packages only have .AURINFO file, but in the future most of the maintained packages will have it.

[antoinealb]

I know this is a bit of an old issue but apparently Puppet supports it since 3.0.0 and uses Yaourt internally with --noconfirm (https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pacman.rb).

Is this still not acceptable for the community to do it ?

[gtmanfred]

Yaourt shouldn't be used, stuff from the aur shouldn't be built from source. What we should be doing is ordering and building straight from provided pkgbuilds, instead of autodoing the dependencies.

[antoinealb]

Maybe we could add a build_untrusted option to indicate this is dangerous, in a similar way to skip_verify which also should not be done. Also, is the biggest problem dependencies or the build code itself ?

[waynew]

@antoinealb I think there are a few issues at play here.

• Within the Arch community the AUR is seen a bit like the Wild West (and I think that's probably accurate). There is no police force, and anyone can pretty much do anything. Granted, most people out there are polite, but it's possible to do nefarious things there.
• Arch specifically states:

There is not and will never be an official mechanism for installing build material from the AUR. All AUR users should be familiar with the build process.

• Many community members don't care and have provided their own tools (mentioned in this thread and elsewhere) that abstract the AUR process. Many people regard this as A Good Thing (tm)
• If Salt develops a method to build from the AUR they take some responsibility. Obviously every admin is responsible for making sure they know what's going on, but technically this is introducing a potential gaping security hole. And I doubt any of us users want that either.

So now we question what's the Right Thing here? Salt tries to be secure by default and make it difficult to make bad choices (see: requiring source_hash on Internet downloads). That's probably not a huge comfort to those of us who want what we want and we want it now :pouting_cat: - but we can always write our own module.

So how do they provide us the feature without saddling themselves with a) gaping security holes and b) a stupid time suck that they have to continue to devote time to?

[antoinealb]

The answer probably depends on what you are doing with Saltstack. If you are managing a fleet of server, of course you don't want to automatically build packages and you probably have the ressources to do so. But if you are managing only one or two server (or desktops) using masterless Saltstack, then it is probably be more convenient to allow the user to build packages from the AUR.

I will probably go down the module route though.

[theblazehen]

Has anyone done anything more on this?

It should be possible to replace the pacman one with a pacaur one simply right?

[gtmanfred]

I don't think we should replace the pacman module to use pacaur.

Using the official package tool for arch is the correct way to go. If someone writes something for pacaur, it should remain gated with a configuration option so people do not get surprised when the non official package manager wrapper is used. On Wed, Jun 8, 2016 at 7:54 AM theblazehen notifications@github.com wrote:

Has anyone done anything more on this?

It should be possible to replace the pacman one with a pacaur one simply right?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/saltstack/salt/issues/10064#issuecomment-224579796, or mute the thread https://github.com/notifications/unsubscribe/AAssocWeiuIyWvhEBEOaEtSPNGWTaOWYks5qJrujgaJpZM4Bd3y- .

[theblazehen]

As I understand it, you could have it as a module, and then users could specify the module that should be used for packages. Or am I wrong there?

[terminalmage]

We'd never make it the default package provider, but if a provider was written for pacaur, it could be set to replace pacman by adding the following to /etc/salt/minion:

providers:
  pkg: pacaur

[waynew]

Does pacaur use pacman? I'm not sure that I would ever want to replace pacman. I would envision aur packages being installed similar to pip installed packages.

Though I do believe aur packages are eventually installed with pacman.

[terminalmage]

I don't know, I don't use it. Personally, I check out the AUR source for a package, build it using makechrootpkg, and then add it to a local repo, where it is eventually installed in a pacman -Syu.

[waynew]

https://bbs.archlinux.org/viewtopic.php?id=119473

Given the description, I'm not sure if it would be ideal here (certainly not as a replacment to pacman).

The way I've been adding AUR packages lately is just doing git clone <whatever/the/aur/readonly/git/link/is>.git && cd <the_folder> && makepkg -si. It seems to work for me so far, though I have had at least one package that required arch packages that I already had pip installed the equivalents (requests, for one).

It's possible that doing something as simple as that might be an appropriate choice. Or as you mention about adding the package to the local repo, that may also be ideal, to have something like:

ttf-google-fonts-git:
    aur:
        - added_locally
    pkg:
        - installed

Obviously added_locally is a garbage name, but something to that effect.

[terminalmage]

Perhaps then a better solution would be a dedicated state and execution module for AUR builds.

[stooj]

I haven't had a chance to look at this properly yet, but I think this project might bypass the issue by making an easy way to maintain your own arch repo:

EnigmaCurry/arch-ppa: Create and maintain personal Arch linux package repositories

[theblazehen]

Yeah, that's what I settled on for now. Would be nice to not need to run that server though

On Thu, Jun 9, 2016 at 11:50 PM, Stoo Johnston notifications@github.com wrote:

I haven't had a chance to look at this properly yet, but I think this project might bypass the issue by making an easy way to maintain your own arch repo:

EnigmaCurry/arch-ppa: Create and maintain personal Arch linux package repositories https://github.com/EnigmaCurry/arch-ppa

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/saltstack/salt/issues/10064#issuecomment-225037427, or mute the thread https://github.com/notifications/unsubscribe/ADH-euB6NF84MG1imrHpCqF7Bm0tK6p8ks5qKIqWgaJpZM4Bd3y- .

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.