Salt tries to merge pkg.installed states even if one has un-met requisites, causing failure.

[racooper]

I have the following state defined:

subscription-manager:
  pkg.installed

satellite-pkg:
  pkg.installed:
    - sources:
      - katello-ca-consumer-satellite-1p-mgmt: http://satellite/pub/katello-ca-consumer-satellite-1p-mgmt-1.0-1.noarch.rpm
    - require:
      - pkg: subscription-manager

{% if salt['grains.get']('subscribed', False) != True %}
{% if 'development' in grains['roles'] %}
  {% set keytype = "Test" %}
{% elif 'production' in grains['roles'] %}
  {% set keytype = "Prod" %}
{% endif %}
subscription-manager register --org="DivIT" --activationkey="CentOS_{{ grains['osmajorrelease'] }}_{{ keytype }}_Key":
  cmd.run:
    - require:
      - pkg: subscription-manager
      - pkg: satellite-pkg

  grains.present:
    - name: subscribed
    - value: True
{% endif %}

RPM-GPG-KEY-katello:
  file.managed:
    - name: /etc/pki/rpm-gpg/RPM-GPG-KEY-katello
    - source: salt://common/katello/RPM-GPG-KEY-katello
    - user: root
    - group: root
    - mode: 644

katello-client_repo:
  pkgrepo.managed:
    - name: katello-client
    - humanname: Katello Client 3.8
    - baseurl: https://fedorapeople.org/groups/katello/releases/yum/3.8/client/el7/$basearch/
    - gpgkey: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-katello
    - enabled: 1
    - gpgcheck: 1
    - require:
      - file: RPM-GPG-KEY-katello

katello-agent:
  pkg.installed:
  - require:
    - pkgrepo: katello-client_repo
    - pkg: satellite-pkg

When the state is run, it tries to merge the two pkg.installed states even though the second one (katello-agent) has requisites that have not been met. So the ultimate output actually wants to install katello-agent twice; first with subscription-manager (which fails, because katello-client-repo has not been run yet), then at the end which also fails, because the initial pkg.install fails since katello-agent fails.

local:
----------
          ID: subscription-manager
    Function: pkg.installed
      Result: False
     Comment: The following packages failed to install/update: katello-agent
              1 targeted package was installed/updated.
     Started: 11:03:51.426704
    Duration: 26386.456 ms
     Changes:   
              ----------
              libnl:
                  ----------
                  new:
                      1.1.4-3.el7
                  old:
              python-dateutil:
                  ----------
                  new:
                      1.5-7.el7
                  old:
              python-dmidecode:
                  ----------
                  new:
                      3.12.2-3.el7
                  old:
              python-ethtool:
                  ----------
                  new:
                      0.8-8.el7
                  old:
              python-syspurpose:
                  ----------
                  new:
                      1.24.13-3.el7.centos
                  old:
              subscription-manager:
                  ----------
                  new:
                      1.24.13-3.el7.centos
                  old:
              subscription-manager-rhsm:
                  ----------
                  new:
                      1.24.13-3.el7.centos
                  old:
              subscription-manager-rhsm-certificates:
                  ----------
                  new:
                      1.24.13-3.el7.centos
                  old:
              usermode:
                  ----------
                  new:
                      1.111-6.el7
                  old:
----------
          ID: satellite-pkg
    Function: pkg.installed
      Result: False
     Comment: One or more requisite failed: common.katello.subscription-manager
     Started: 11:04:17.875041
    Duration: 0.017 ms
     Changes:   
----------
          ID: subscription-manager register --org="TAMUCS" --activationkey="CentOS_7_Prod_Key"
    Function: cmd.run
      Result: False
     Comment: One or more requisite failed: common.katello.satellite-pkg, common.katello.subscription-manager
     Started: 11:04:17.876426
    Duration: 0.013 ms
     Changes:   
----------
          ID: subscription-manager register --org="TAMUCS" --activationkey="CentOS_7_Prod_Key"
    Function: grains.present
        Name: subscribed
      Result: True
     Comment: Set grain subscribed to True
     Started: 11:04:17.876867
    Duration: 18.226 ms
     Changes:   
              ----------
              subscribed:
                  True
----------
          ID: RPM-GPG-KEY-katello
    Function: file.managed
        Name: /etc/pki/rpm-gpg/RPM-GPG-KEY-katello
      Result: True
     Comment: File /etc/pki/rpm-gpg/RPM-GPG-KEY-katello updated
     Started: 11:04:17.897121
    Duration: 96.374 ms
     Changes:   
              ----------
              diff:
                  New file
              mode:
                  0644
----------
          ID: katello-client_repo
    Function: pkgrepo.managed
        Name: katello-client
      Result: True
     Comment: Configured package repo 'katello-client'
     Started: 11:04:17.994944
    Duration: 1502.673 ms
     Changes:   
              ----------
              repo:
                  katello-client
----------
          ID: katello-agent
    Function: pkg.installed
      Result: False
     Comment: One or more requisite failed: common.katello.satellite-pkg
     Started: 11:04:19.498295
    Duration: 0.013 ms
     Changes:   
Summary for local
------------
Succeeded: 3 (changed=4)
Failed:    4
------------
Total states run:     7
Total run time:  28.004 s

This is using Salt 2018.3 on CentOS 7.

[garethgreenaway]

@racooper Thanks for the report. Are you by chance using the aggregation feature with pkg state? Can you see if you have the state_aggregate: True in the minion configuration? Thanks!

[racooper]

In configuration, it's set to: #state_aggregate: False in both minion and master configs, which I presume is the default. It's not a setting I changed.

[racooper]

We found that our salt master config has enabled:

state_aggregate:
  - pkgs

it was in a different part of the config file than the commented out default. We found that disabling that setting does resolve the reported issue, but states take noticeably longer to run (which makes sense). I still think it's a bug to merge pkg states if there are unmet dependencies.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.