How to cause salt to use apt-fast if it's installed?

[thedrow]

apt-fast downloads Debian packages in parallel and saves them into the apt cache folder so that they are installed from cache. If I use the pkg state Salt will use the apt-get executable. I'm not sure that pkg state should be changed to support apt-fast but there must be a solution to use it with the pkg state. Should I alias apt-fast to the apt-get command? Will that work?

[jfindlay]

@thedrow, this is something that could be implemented in salt in the aptpkg module. You can also define /usr/local/bin/apt-get as

#!/bin/sh
apt-fast "${@}"

Using an alias probably won't work since salt usually bypasses shell modifications to the environment and as of 2014.7.2 salt doesn't use a shell by default. Debian may also have a builtin way of doing this for you with /etc/alternatives or something.

[techhat]

@jfindlay, is it possible for him to set it as a pkg provider?

[thedrow]

I think that apt-fast itself is useful but do we want to provide support for using it from within Salt or document how to cause Salt to use it?

[jfindlay]

@thedrow, I'm persuaded to not support it directly within salt, but we can write documentation on how to set it up if you want. It seems like a package manager addon/frontend that is best suited to be configured and activated from within the system itself. Salt should probably be using the system's package manager in a standard manner, irrespective of how it is implemented or extended.

[jfindlay]

Or, as @techhat suggests, you could set it as the pkg provider for your minion in your minion config. This may be the best solution yet.

[thedrow]

Yup. We just need to discover a way that will ensure that Salt will pick apt-fast as apt-get and document it. I have no clue which is why I opened this issue in the first place.

[jfindlay]

Great. To me, it seems like setting it as the pkg provider in the minion config is the best way to go. Forgive my label juggling.

[ag-wood]

@jfindlay, the package provider setting in the minion config depends on there being a corresponding package module. I don't believe this answers the original question.

The original suggestion for adding a script called apt-get is a feasible option so long as apt-fast.conf is updated with the fully qualified path to apt-get (i.e. /usr/bin/apt-get) , as apt-fast calls apt-get from the OS path by default.

So the doco could include the steps (after testing):

• Add the _APTMGR setting in /etc/apt-fast.conf with the full path to agt-get:

_APTMGR=/usr/bin/apt-get

• Add a script to over-ride apt-get into /usr/local/bin: (from above)

#!/bin/sh
apt-fast "${@}"

Of course, the above could be established by a salt state that runs before packages are installed.

[thedrow]

I just tried what @ag-wood suggested and it doesn't work. Here's my $PATH:

/usr/lib/ccache:/home/omer/.rbenv/shims:/home/omer/.rbenv/bin:/home/omer/.pyenv/shims:/home/omer/.pyenv/bin:/home/omer/.local/bin:./node_modules/.bin:/usr/lib/ccache:/home/omer/.rbenv/shims:/home/omer/.rbenv/bin:/home/omer/.pyenv/shims:/home/omer/.pyenv/bin:/home/omer/.local/bin:./node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/omer/.gem/ruby/1.8/bin:/opt/nginx/sbin:/home/omer/.fzf/bin:/home/omer/.gem/ruby/2.2.0/bin:/home/omer/.gem/ruby/1.8/bin:/opt/nginx/sbin:/home/omer/.gem/ruby/2.2.0/bin

[thedrow]

Oh you should have mentioned that chmod +x is required. It works now.

[jfindlay]

@ag-wood, creating a script named apt-get that calls apt-fast seems like a workaround hack to me. I think the real solution would be to create a pkg module and use the minion config for pkg provider. Or even have salt try to detect apt-fast on the system and automatically prefer it over apt-get. What do you think?

[ag-wood]

@jfindlay, yes, I agree that the script seems like a workaround but a separate provider would not be optimum either in my thinking. It's not really a new provider, it's functionally equivalent to the apt provide and would need to be a copy of the same.

After looking more at the aptpkg.py provider, it appears to be a simple matter to add the functionality to that provider rather than duplicating the entire functionality in another module.

I created an example of this against the 2015.2 branch on my fork at: https://github.com/ag-wood/salt/commit/17f800deb9cca0888800c6d7c0c67ee3d501d187

To use the new functionality requires the use_aptfast option for a package state, as in:

vim-tiny:
  pkg.installed:
    - use_aptfast: True

Alternatively, we could just remove the additional option and prefer apt-fast if installed (I'm not sure what the protocol is for making assumptions about using a wrapper just because it's installed). @thedrow - any comment on what the preferred behaviour would be?

I can create a pull request for the above if you like - should that be against develop or 2015.2 branch?

[jfindlay]

That looks like a good start. I don't think we should encumber the salt configs with platform-specific stuff. If promoting apt-fast to full pkg provider status is too duplicative, then your proposed solution might be the best way to go.

We could also abstract the apt module for the name of the executable, apt-get or apt-fast so that they share the same common code, but considering how large, complex, and important that module is, it may not be worth the effort. I'm starting to think that your proposal looks like the right idea.

[thedrow]

I'm not sure what the default behavior should be but I think that any Debian based distribution users can get value out of apt-get.

[thedrow]

After thinking about it a bit, I'd rather see this in the minion's configuration file. It makes less sense to incorporate it into the state itself.

[ag-wood]

Continuing then...

There is a section in the minion configuration file that describes the possibility of adding ad-hoc 'module specific' settings as per:

######      Module configuration      #####
###########################################
# Salt allows for modules to be passed arbitrary configuration data, any data
# passed here in valid yaml format will be passed on to the salt minion modules
# for use. It is STRONGLY recommended that a naming convention be used in which
# the module name is followed by a . and then the value. Also, all top level
# data must be applied via the yaml dict construct, some examples:
#
# You can specify that all modules should run in test mode:
#test: True
#
# A simple value for the test module:
#test.foo: foo
#
# A list for the test module:
#test.bar: [baz,quo]
#
# A dict for the test module:
#test.baz: {spam: sausage, cheese: bread}

I propose, in addition to the state option to invoke apt-fast, we include another option in the minion configuration to enable apt-fast on a global basis at the minion.

The setting would be aptpkg.use_aptfast: True (Note: Adding this requires a restart of the minion).

The updated code looks like: https://github.com/ag-wood/salt/commit/2c11971734e34f23de0b0e1aeea59b53adeedddd

@jfindlay, @thedrow - thoughts?

[thedrow]

Sounds good but note that apt-fast should be installed for this to work.

[ag-wood]

Yes - the code checks that the command is present in the path and that one of the options are set to True. On 20/04/2015 10:28 PM, "Omer Katz" notifications@github.com wrote:

Sounds good but note that apt-fast should be installed for this to work.

— Reply to this email directly or view it on GitHub https://github.com/saltstack/salt/issues/22557#issuecomment-94438809.

[jfindlay]

@ag-wood, seems good to me, thanks.

[adamfeuer]

@jfindlay @thedrow @ag-wood Hi guys, are you still interested in adding support for apt-fast? If so, I updated @ag-wood's work to the latest salt develop, you can see it here:

https://github.com/saltstack/salt/compare/develop...adamfeuer:apt-fast

( @ag-wood sorry I couldn't figure out how to base my branch on your commits, since aptpkg.py code has moved on a bit from the point where you made the modifications. So I copied the code from your branch and refactored it a bit. )

If you are still interested and like this - I can add documentation and file a pull request. If you have improvements I would like to know them too!

[thedrow]

Maybe that will be the best course of action. @jfindlay I'm undecided on this one. apt-fast is useful just because it makes provisioning servers faster. If that's a goal to you guys, let's merge @adamfeuer's PR.

[adamfeuer]

@thedrow This change is useful to my team, because it shortens our development cycle - we use Salt to build many Docker images, so substantially increasing the speed of package installationl helps us have a sane development environment.

[adamfeuer]

@thedrow @jfindlay I'll make a PR for this later today so you guys can decide on it.

[jfindlay]

@adamfeuer, that would be great, thanks, although I'm still unsure why this wouldn't work with the provider config. This is a minor complaint, though.

[adamfeuer]

@jfindlay Hmmm, I just didn't try that. Are you suggesting writing an aptfast provider that would derive from aptpkg, and then configure it to be used instead of aptpkg?

[jfindlay]

@adamfeuer, probably adding it as an option to the apt module should be fine for now. There is no reason to hold out for a full module if most or all of the advantage can be provided with some minor additions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.