saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
13.98k stars 5.47k forks source link

[BUG] network.interface on windows 10 minions returns (WMI?) interface description instead of name. #57484

Open r-pufky opened 4 years ago

r-pufky commented 4 years ago

Description Expected interface name is not returned for network.interface on Windows 10 salt clients.

Intended result: Have minions contact salt-master via wireguard tunnel, named "salt-client". This network is confirmed to: work, pingable on both minion/master machines and correctly applies salt states, minion client upgrades, etc. through the tunnel without specifying the interface, salt-minions will use the correct network adapter to contact the salt-master over wireguard (working).

However, explicitly specifying the interface name will cause windows minions to fail.

Linux

source_interface_name: salt-client

Working as intended. Properly identifies the salt-client wireguard tunnel and uses it.

Windows

source_interface_name: salt-client

Will report that the interface does not exist.

windows salt-client interface retrieval.

 .\salt-call.bat network.interface salt-client
[ERROR   ] Interface "salt-client" not in available interfaces: "Software Loopback Interface 1", "Intel(R) Ethernet Connection (2) I219-V", "Wintun Userspace Tunnel"
local:
    Interface "salt-client" not in available interfaces: "Software Loopback Interface 1", "Intel(R) Ethernet Connection (2) I219-V", "Wintun Userspace Tunnel"

powershell reports interface names appropriately.

Get-NetAdapter
Name                      InterfaceDescription                    ifIndex Status       MacAddress        LinkSpe
                                                                                                              ed
----                      --------------------                    ------- ------       ----------        -------
Ethernet 2                Adapter for Windows                          19 Up           AA-BB-CC-DD-EE-FF ...Mbps
Ethernet                  Intel(R) Ethernet Connection (2) I219-V       5 Up           AA-BB-CC-DD-EE-FF  1 Gbps
salt-client               WireGuard Tunnel                              3 Up                             ...Gbps

Control Panel reports properly as well.

salt-bug

Detailed interface list shows

salt-details This shows that saltstack is pulling the interface from the interface description (I'm guessing WMI, as the description is different from even the powershell listing).

A clear and concise description of what the bug is.

Expected interface name is not returned for network.interface on Windows 10 salt clients.

Expected Name: salt-client Returned Name: Wintun Userspace Tunnel

This value is used in salt minion source_interface_name and diverges from linux minions.

Setup

(Please provide relevant configs and/or SLS files (be sure to remove sensitive info).

See above.

Steps to Reproduce the behavior

(Include debug logs if possible and relevant)

  1. Setup a working wireguard network between minion and master (I can send my wireguard formula, but that is relatively complicated and not the core issue here).
  2. Confirm minion and master can communicate and apply states over wireguard network without needing to specify the adapter name, on a private IP address space (/24).
  3. Explicitly set network adapter name in config:

c:/salt/minion

source_interface_name: salt-client
  1. Run state.highstate or ./salt-call network.interface salt-client

Expected behavior

A clear and concise description of what you expected to happen.

The same behavior as linux.

network.interface salt-client returns the correct interface. source_interface_name: salt-client is valid for windows salt minions properly configured.

Versions Report

salt-master

Salt Version:
           Salt: 3000.3

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: 2.1.11
         Jinja2: 2.10
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: 3.6.1
         pygit2: Not Installed
         Python: 3.7.3 (default, Dec 20 2019, 18:57:59)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 17.1.2
          smmap: 2.0.5
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.1

System Versions:
           dist: debian 10.4
         locale: UTF-8
        machine: x86_64
        release: 4.19.0-9-amd64
         system: Linux
        version: debian 10.4

salt-minion (windows)

Salt Version:
           Salt: 3000.2

Dependency Versions:
           cffi: 1.12.2
       cherrypy: 17.4.1
       dateutil: 2.8.0
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: Not Installed
   pycryptodome: 3.8.1
         pygit2: Not Installed
         Python: 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)]
   python-gnupg: 0.4.4
         PyYAML: 5.1.2
          PyZMQ: 18.0.1
          smmap: 2.0.5
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.1

System Versions:
           dist:
         locale: cp1252
        machine: AMD64
        release: 10
         system: Windows
        version: 10 10.0.18362 SP0 Multiprocessor Free

salt-minion (linux)

Salt Version:
           Salt: 3000.3

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.9 (default, Apr 18 2020, 01:56:04)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 5.3.0-53-generic
         system: Linux
        version: Ubuntu 18.04 bionic

Additional context Add any other context about the problem here.

Happy to upgrade windows minion to 3000.3; but that's a red herring, giving the return value context for the windows minion.

edit: provided the right versions report for windows client.

Akm0d commented 4 years ago

I solved a similar issue in idem-windows a while back, the code in idem-windows might be helpful for finding a solution

xeacott commented 4 years ago

@r-pufky Thanks for so much detail in the report. Ill get started on a fix for this.

xeacott commented 4 years ago

So this boils down to salt see's the interface name as Wintun Userspace Tunnel, on Windows... But on linux, salt see's it as salt-client?

And both systems are using source_interface_name: salt-client

xeacott commented 4 years ago

It also looks like in minion.py there is a check for

if opts['source_interface_name']:
        log.trace("Custom source interface required: %s", opts["source_interface_name"])
        interfaces = salt.utils.network.interfaces()
        log.trace("The following interfaces are available on this Minion:")
        log.trace(interfaces)

and there are log.trace messages saying that if it can't connect to what was provided, the minion log should have this message in it:

        else:
            log.warning(
                "%s is not a valid interface. Ignoring.", opts["source_interface_name"]
            )

Would you be able to look at see what in in your minion on startup?

r-pufky commented 4 years ago

So this boils down to salt see's the interface name as Wintun Userspace Tunnel, on Windows... But on linux, salt see's it as salt-client?

And both systems are using source_interface_name: salt-client

Correct.

It also looks like in minion.py there is a check for

if opts['source_interface_name']:
        log.trace("Custom source interface required: %s", opts["source_interface_name"])
        interfaces = salt.utils.network.interfaces()
        log.trace("The following interfaces are available on this Minion:")
        log.trace(interfaces)

and there are log.trace messages saying that if it can't connect to what was provided, the minion log should have this message in it:

        else:
            log.warning(
                "%s is not a valid interface. Ignoring.", opts["source_interface_name"]
            )

Would you be able to look at see what in in your minion on startup?

I clipped the trace log to the relevant information.

[DEBUG   ] Connecting to master. Attempt 1 (infinite attempts)
[TRACE   ] Custom source interface required: salt-client
[TRACE   ] The following interfaces are available on this Minion:
[TRACE   ] {'WireGuard Tunnel': {'up': True, 'inet': [{'label': 'WireGuard Tunnel', 'gateway': '', 'address': '172.31.255.10', 'broadcast': '172.31.255.255', 'netmask': '255.255.255.0'}], 'hwaddr': ':::::'}, 'Intel(R) Ethernet Connection (2) I219-V': {'up': True, 'inet6': [{'address': '{REDACTED}', 'gateway': ''}], 'inet': [{'label': 'Intel(R) Ethernet Connection (2) I219-V', 'gateway': '{REDACTED}', 'address': '{REDACTED}', 'broadcast': '{REDACTED}', 'netmask': '{REDACTED}'}], 'hwaddr': '{REDACTED}'}, 'Software Loopback Interface 1': {'up': True, 'inet6': [{'address': '::1', 'gateway': ''}], 'inet': [{'label': 'Software Loopback Interface 1', 'gateway': '', 'address': '127.0.0.1', 'broadcast': '127.255.255.255', 'netmask': '255.0.0.0'}], 'hwaddr': ':::::'}}
[WARNING ] salt-client is not a valid interface. Ignoring.
[DEBUG   ] Master URI: tcp://172.31.255.254:4506
PS C:\salt>  .\salt-call.bat network.interface salt-client
[ERROR   ] Interface "salt-client" not in available interfaces: "Software Loopback Interface 1", "Intel(R) Ethernet Connection (2) I219-V", "WireGuard Tunnel"
local:
    Interface "salt-client" not in available interfaces: "Software Loopback Interface 1", "Intel(R) Ethernet Connection (2) I219-V", "WireGuard Tunnel"
Get-NetAdapter
Name                      InterfaceDescription                    ifIndex Status       MacAddress        LinkSpe
                                                                                                              ed
----                      --------------------                    ------- ------       ----------        -------
Ethernet 2                Adapter for Windows                          19 Up           AA-BB-CC-DD-EE-FF ...Mbps
Ethernet                  Intel(R) Ethernet Connection (2) I219-V       5 Up           AA-BB-CC-DD-EE-FF  1 Gbps
salt-client               WireGuard Tunnel                              3 Up                             ...Gbps

Let me know if you need anything else!

xeacott commented 4 years ago

Thanks for getting all that information for me 😄 Going down the rabbit hole of all this eventually leads to this function:

def get_interface_info():
    """
    This function will return network interface information for the system and
    will use the best method to retrieve that information. Windows 7/2008R2 and
    below will use WMI. Newer systems will use .NET.
    Returns:
        dict: A dictionary of information about the Network interfaces
    """
    # On Windows 7 machines, use WMI as dotnet 4.0 is not available by default
    if USE_WMI:
        return get_interface_info_wmi()
    else:
        return get_interface_info_dot_net_formatted()

Since it appears you're on a Win10 box (likewise here, too) you'll actually be using the python dotnet framework instead of WMI because of this...

if IS_WINDOWS:
    USE_WMI = StrictVersion(platform.version()) < StrictVersion("6.2")
    if USE_WMI:
        import wmi
        import salt.utils.winapi
    else:
        import clr
        from System.Net import NetworkInformation

Just keeping all this info here... the rabbit hole for gathering network interfaces goes like 10 calls down, phew!

xeacott commented 4 years ago

Oh, finally could you run a salt-call --local network.interfaces for me and see what you get? Looks like we only count a network interface if its status is "Up", and then the "name" that we give each interface is actually the description itself: name = interfaces[i_face]['description'] See if this PR works for you...https://github.com/saltstack/salt/pull/57633

r-pufky commented 4 years ago

No worries. Here you are:

C:\salt> .\salt-call.bat --local network.interfaces
local:
    ----------
    Intel(R) Ethernet Connection (2) I219-V:
        ----------
        hwaddr:
            {REDACTED}
        inet:
            |_
              ----------
              address:
                  {REDACTED}
              broadcast:
                  {REDACTED}
              gateway:
                  {REDACTED}
              label:
                  Intel(R) Ethernet Connection (2) I219-V
              netmask:
                  {REDACTED}
        inet6:
            |_
              ----------
              address:
                  {REDACTED}
              gateway:
        up:
            True
    Software Loopback Interface 1:
        ----------
        hwaddr:
            :::::
        inet:
            |_
              ----------
              address:
                  127.0.0.1
              broadcast:
                  127.255.255.255
              gateway:
              label:
                  Software Loopback Interface 1
              netmask:
                  255.0.0.0
        inet6:
            |_
              ----------
              address:
                  ::1
              gateway:
        up:
            True
    WireGuard Tunnel:
        ----------
        hwaddr:
            :::::
        inet:
            |_
              ----------
              address:
                  172.31.255.10
              broadcast:
                  172.31.255.255
              gateway:
              label:
                  WireGuard Tunnel
              netmask:
                  255.255.255.0
        up:
            True

Since it appears you're on a Win10 box (likewise here, too)

Correct. Here's the grain info for Windows 10:

    os:
        Windows
    os_family:
        Windows
    osfinger:
        Windows-10
    osfullname:
        Microsoft Windows 10 Pro
    osmanufacturer:
        Microsoft Corporation
    osrelease:
        10
    osrelease_info:
        - 10
    osservicepack:
        None
    osversion:
        10.0.18363
I3urny commented 1 year ago

Any particular reason #57633 got closed without a comment? This is still an issue in v3006 and it seems to me as if that was the correct solution. I'd be happy help resolve this issue but I can't imagine there's much missing as @xeacott has done 99% of the work already.

v3006 behaviour:

> salt-call --local network.interfaces
local:
    ----------
    Intel(R) Ethernet Connection (16) I219-V:
        ----------
        hwaddr:
            {REDACTED}
        inet:
            |_
              ----------
              address:
                  {REDACTED}
              broadcast:
                  {REDACTED}
              gateway:
                  {REDACTED}
              label:
                  Intel(R) Ethernet Connection (16) I219-V
              netmask:
                  {REDACTED}
        inet6:
            |_
              ----------
              address:
                  {REDACTED}
              gateway:
              prefixlen:
                  64
        up:
            True
    Software Loopback Interface 1:
        ----------
        hwaddr:
            :::::
        inet:
            |_
              ----------
              address:
                  127.0.0.1
              broadcast:
                  127.255.255.255
              gateway:
              label:
                  Software Loopback Interface 1
              netmask:
                  255.0.0.0
        inet6:
            |_
              ----------
              address:
                  ::1
              gateway:
              prefixlen:
                  128
        up:
            True
    WireGuard Tunnel:
        ----------
        hwaddr:
            :::::
        inet:
            |_
              ----------
              address:
                  10.0.0.2
              broadcast:
                  10.0.0.2
              gateway:
              label:
                  WireGuard Tunnel
              netmask:
                  255.255.255.255
        up:
            True

Using alias instead of description as described in #57633:

> salt-call --local network.interfaces
local:
    ----------
    Ethernet:
        ----------
        hwaddr:
            {REDACTED}
        inet:
            |_
              ----------
              address:
                  {REDACTED}
              broadcast:
                  {REDACTED}
              gateway:
                  {REDACTED}
              label:
                  Ethernet
              netmask:
                  {REDACTED}
        inet6:
            |_
              ----------
              address:
                  {REDACTED}
              gateway:
              prefixlen:
                  64
        up:
            True
    Loopback Pseudo-Interface 1:
        ----------
        hwaddr:
            :::::
        inet:
            |_
              ----------
              address:
                  127.0.0.1
              broadcast:
                  127.255.255.255
              gateway:
              label:
                  Loopback Pseudo-Interface 1
              netmask:
                  255.0.0.0
        inet6:
            |_
              ----------
              address:
                  ::1
              gateway:
              prefixlen:
                  128
        up:
            True
    wg0:
        ----------
        hwaddr:
            :::::
        inet:
            |_
              ----------
              address:
                  10.0.0.2
              broadcast:
                  10.0.0.2
              gateway:
              label:
                  wg0
              netmask:
                  255.255.255.255
        up:
            True