[BUG] Minion behind NAT incorrectly shows not connected

Description I have a minion on my local network connecting to a cloud-based master. That minion never shows as connected, but all my minions with public IP addresses do show as connected. That is, the local minion doesn't show as connected in SaltGUI, nor show up in the results of salt-run manage.present. However, salt functions still work because it is actually connected.

I did some digging around in the Salt source code and discovered that the problem is in the way Salt determines whether a minion is connected. It works like this:

Get the list of known minions.
Run ss -ant src <master_publish_port> to get a list of active network connections to the master's port.
Compare the list of active connections with each minion's reported IP addresses (like running ip address on the minion).
If one of the minion's reported IP addresses is in the list of connections, it must be connected.
If not, then it is assumed disconnected.

The problem is that when a minion is behind NAT, it doesn't know its public IP address, and therefore is never connected according to this algorithm. A much better way to do it would be for the Salt master to keep a list of the minions that it received a connection from. However I am not familiar with ZeroMQ so maybe this is impossible?

Setup

Please be as specific as possible and give set-up details.

[x] on-prem machine
[ ] VM (Virtualbox, KVM, etc. please specify)
[ ] VM running on a cloud service, please be explicit and add details
[ ] container (Kubernetes, Docker, containerd, etc. please specify)
[ ] or a combination, please be explicit
[ ] jails if it is FreeBSD
[x] classic packaging
[ ] onedir packaging
[x] used bootstrap to install

Steps to Reproduce the behavior salt-run manage.present is showing

- (all minions except fs2)

Expected behavior salt-run manage.present should show

- fs2
- (other minions)

Screenshots

Versions Report

Salt Version:
          Salt: 3007.0

Python Version:
        Python: 3.10.13 (main, Feb 19 2024, 03:31:20) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.3
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.15.1
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4

Salt Package Information:
  Package Type: onedir

System Versions:
          dist: debian 12.5 bookworm
        locale: utf-8
       machine: x86_64
       release: 6.1.0-18-amd64
        system: Linux
       version: Debian GNU/Linux 12.5 bookworm

Additional context

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

saltstack / salt

[BUG] Minion behind NAT incorrectly shows not connected #66295