[BUG] 3006.9 master not able to target minion by grain correctly

[tjyang]

Description For example, salt -G "os:macos" test.ping will display correct output of macos minions but right after the correct display it also display other minions and make them as not responded in time(salt-run jobs.lookup_jid xxxxx). I believe this is a regression from early version of salt.

Setup (Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)

Please be as specific as possible and give set-up details.

• [ x] on-prem machine
• [x] VM (Virtualbox, KVM, etc. please specify)
• [ ] VM running on a cloud service, please be explicit and add details
• [ ] container (Kubernetes, Docker, containerd, etc. please specify)
• [ ] or a combination, please be explicit
• [ ] jails if it is FreeBSD
• [ ] classic packaging
• [ x] onedir packaging
• [ x] used bootstrap to install

Steps to Reproduce the behavior Ex: two real macos minions responded but it also match the other 300 non-macos minions.

[me@salt01]$ sudo salt -G "os:macos" test.ping
<snipped>
-------------------------------------------
Summary
-------------------------------------------
# of minions targeted: 300
# of minions returned: 2
# of minions that did not return: 300
# of minions with errors: 0
-------------------------------------------
ERROR: Minions returned with non-zero exit code
[me@salt01]$

• this is rocklinux case: 200 non rocky minion also got contacted for ping result and failed.

  [me@salt01 ]$ sudo salt -C 'G@os:rocky' test.ping
  <snipped>
  -------------------------------------------
  Summary
  -------------------------------------------
  # of minions targeted: 400
  # of minions returned: 200
  # of minions that did not return: 200
  # of minions with errors: 0
  -------------------------------------------
  ERROR: Minions returned with non-zero exit code
  [me@salt01 ]$ 

Expected behavior A clear and concise description of what you expected to happen.

Screenshots

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ```yaml Salt Version: Salt: 3006.9 Python Version: Python: 3.10.14 (main, Jun 26 2024, 11:44:37) [GCC 11.2.0] Dependency Versions: cffi: 1.14.6 cherrypy: unknown cryptography: 42.0.5 dateutil: 2.8.1 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 3.1.4 libgit2: Not Installed looseversion: 1.0.2 M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.2 msgpack-pure: Not Installed mysql-python: Not Installed packaging: 22.0 pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.19.1 pygit2: Not Installed python-gnupg: 0.4.8 PyYAML: 6.0.1 PyZMQ: 23.2.0 relenv: 0.17.0 smmap: Not Installed timelib: 0.2.4 Tornado: 4.5.3 ZMQ: 4.3.4 System Versions: dist: rocky 8.10 Green Obsidian locale: utf-8 machine: x86_64 release: 4.18.0-553.16.1.el8_10.x86_64 system: Linux version: Rocky Linux 8.10 Green Obsidian ```

Additional context

• Reference:
  • https://github.com/saltstack/salt/issues/66774
  • https://github.com/saltstack/salt/issues/57750

[ewaldmire]

FYI: https://github.com/saltstack/salt/pull/66973

[tjyang]

FYI: #66973 Thanks for the link, but it is about removing macos build infra server. not about salt-master's issue using os grain to poke minions.

[tjyang]

• Issue 1: Minion need to be restarted.

  salt -G 'os:redhat' test.ping # should returns minons that are redhat

  Workaround solution by restarting unnecessary targeted minions .

Restarting the windows minions.
after the salt-master upgrade cleared the odd unnecessary targeting we were seeing.

• Issue 2: If a host is down at time of running grain targetting the salt -G 'os:redhat' test.ping will failed and showing all those disconnected minion name.