search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.07k stars 5.47k forks source link

[FEATURE REQUEST] Add flag to pkg.installed to force dpkg usage instead of apt #65315

Open devin-cpg opened 11 months ago

devin-cpg commented 11 months ago

Existing Problem: After updating from 3003.5 to 3006.3, we experienced failure to install some packages that were sourced from the salt server, as well as one from an https:// repository. These failures all seem to stem from the change from dpkg to apt-get as seen in #62934 and #63126

First, the storcli .deb package, which lives on the salt master:

Function: pkg.installed
  Result: False
 Comment: Problem encountered installing package(s). Additional info follows:

          errors:
              - Running scope as unit: run-red1bb222c4c2490197650686c372ab6a.scope
                E: Problem parsing Provides line of storcli:all=007.2002.0000.0000
                E: Error occurred while processing storcli (NewVersion2)
                E: Problem with MergeList /var/cache/salt/minion/files/base/files/pkgs/storcli_007.2002.0000.0000_all.deb
                E: The package lists or status file could not be parsed or opened.
 Started: 10:28:05.898741
Duration: 2603.801 ms
 Changes:

Next would be the sumologic collector. The state is extremely simple:

install sumocollector:
  pkg.installed:
    - sources: 
      - sumocollector: https://collectors.sumologic.com/rest/download/deb/64

However, in 3006 this fails:

errors:
    - Running scope as unit: run-rf1a49f307b2141348111e17aedfa5ea9.scope
      E: Unsupported file /var/cache/salt/minion/extrn_files/base/collectors.sumologic.com/rest/download/deb/64 given on commandline
[ERROR   ] Parent directory not present
local:
----------
          ID: install sumocollector
    Function: pkg.installed
      Result: False
     Comment: Problem encountered installing package(s). Additional info follows:

              errors:
                  - Running scope as unit: run-rf1a49f307b2141348111e17aedfa5ea9.scope
                    E: Unsupported file /var/cache/salt/minion/extrn_files/base/collectors.sumologic.com/rest/download/deb/64 given on commandline
     Started: 10:42:39.815159
    Duration: 9146.275 ms
     Changes:
----------

Requested Solution: A parameter for the pkg.installed (or rather, I guess salt.modules.aptpkg.install) to revert back to the old pre #62934 behavior and just use dpkg instead of apt-get.

Alternatives: With regards to the storcli package, this was resolved by unpacking, deleting the Provides: line from the control file, and repacking. It seems that Broadcom is not building these properly, or at least not well enough for apt to install.

The sumologic collector requires a change to the state where instead of the simple version as seen above, I need to use a chain of cmd.run to download and install.

welcome[bot] commented 11 months ago

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

devin-cpg commented 11 months ago

For the sumologic collector, if the Content-Disposition header could be used to generate the filename I suspect this would work as it previously did. Using wget --content-disposition https://collectors.sumologic.com/rest/download/deb/64 returns (as of 2023-09-29) sumocollector_19.456-3_amd64.deb when run from a 20.04 focal box.

EDIT: Just doing some light research, it looks like tornado.httputil.parse_multipart_form_data can return the filename from the Content-Disposition header.

dmurphy18 commented 9 months ago

@devin-cpg Salt 3006 and up use OneDir architecture with all dependencies already built-in, and do not rely on OS packages. If you want Salt 3006 to install other dependencies, that you need to install them, it is not automatic.

The OneDir architecture was developed to be self-contained and not rely on OS packages since various OS, are not keeping up with dependency releases and use EOL'd Python, e.g. RedHat 7 & 8 still Python 3.6. Hence in order to allow Salt to move forward, the Salt packages not are self-contained and do not use OS provided packages. The user has to pip install those dependencies they also desire, using salt-pip.

Closing this since working as designed for Salt 3006 and up. But feel free to reopen if there are other points to raise.

dmurphy18 commented 9 months ago

@devin-cpg Rereading initial issue, could be unrelated to OneDir and wanting just packages installed on Debian family OS, and OneDir favoring apt over dpkg which I quite rightly understand. As for the issues mentioned, that 3005.1 ubuntu 22.04 was a Tiamat self.contained Salt package, very similar to OneDir (Relenv backed instead of Tiamat which had some issues). Leaving as feature request