Protocol 3.5 Device Discovery Updates

[jasonacox]

Device discovery packets redux

Newer v3.5 devices do not send out unsolicited discovery broadcasts. Instead, they listen for broadcasts from a client app and send their discovery packet directly to that client.

The client broadcast contains the payload {"from":"app","ip":"192.168.1.42"} and is sent to the subnet broadcast address (i.e. 192.168.1.255) on port 7000. It is GCM encrypted the same way as broadcasts above.

When a device receives a client broadcast, it responds by sending a device discovery packet directly to the IP address specified in the client broadcast. This again uses port 7000 and is GCM encrypted the same way as broadcasts above. The device I used in my testing will send the device discovery packet out as a broadcast if the client app specifies "255.255.255.255" as the IP address, however I suspect this is an accident and is not something they intended to be used.

Originally posted by @uzlonewolf in https://github.com/jasonacox/tinytuya/discussions/260#discussioncomment-9979656

[jasonacox]

I wonder if we could use a function to pull the local IPs and broadcast addresses. This would require psutil or netifaces:

import importlib

def get_ip_to_broadcast():
    if importlib.util.find_spec('psutil'):
        import psutil
        import socket

        interfaces = psutil.net_if_addrs()
        ip_to_broadcast = {}

        for addresses in interfaces.values():
            for addr in addresses:
                if addr.family == socket.AF_INET and addr.broadcast:  # AF_INET is for IPv4
                    ip_to_broadcast[addr.address] = addr.broadcast

        return ip_to_broadcast

    elif importlib.util.find_spec('netifaces'):
        import netifaces

        interfaces = netifaces.interfaces()
        ip_to_broadcast = {}

        for interface in interfaces:
            addresses = netifaces.ifaddresses(interface)
            ipv4 = addresses.get(netifaces.AF_INET)

            if ipv4:
                for addr in ipv4:
                    if 'broadcast' in addr:
                        ip_to_broadcast[addr['addr']] = addr['broadcast']

        return ip_to_broadcast

    else:
        raise ImportError("Neither psutil nor netifaces is installed. Please install one of these packages to proceed.")

# Example usage:
print(get_ip_to_broadcast())

[uzlonewolf]

That looks like it would work. The scanner already uses netifaces if it is installed to get the force-scan list.

Instead of raising an error if neither are installed, we could also use the getmyIP() method of just making a random connection and seeing what IP was used and just bind to it and broadcast to 255.255.255.255. It's not going to work on multi-interface machines but would at least work on single-interface ones (which I suspect is what most users have anyway).

[uzlonewolf]

Hmm, I just tried it on one of my single board computers but it blew up with

$ python3
Python 3.11.2 (main, May  2 2024, 11:59:08) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib
>>> importlib.util.find_spec('netifaces')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'importlib' has no attribute 'util'

It seems importlib.util is separate from importlib https://discuss.python.org/t/python3-11-importlib-no-longer-exposes-util/25641 and I'm not sure which python version added importlib.util.

[uzlonewolf]

Ok, this is what I'm planning on adding to the scanner. I flipped addr and broadcast in the ip_to_broadcast dict so interfaces with multiple IPs in the same subnet are only added once.

...
try:
    import psutil # pylint: disable=E0401
    PSULIBS = True
except ImportError:
    PSULIBS = False

...

def get_ip_to_broadcast():
    ip_to_broadcast = {}

    if NETIFLIBS:
        interfaces = netifaces.interfaces()
        for interface in interfaces:
            addresses = netifaces.ifaddresses(interface)
            ipv4 = addresses.get(netifaces.AF_INET)

            if ipv4:
                for addr in ipv4:
                    if 'broadcast' in addr and 'addr' in addr and addr['broadcast'] != addr['addr']:
                        ip_to_broadcast[addr['broadcast']] = addr['addr']

        if ip_to_broadcast:
            return ip_to_broadcast

    if PSULIBS:
        interfaces = psutil.net_if_addrs()
        for addresses in interfaces.values():
            for addr in addresses:
                if addr.family == socket.AF_INET and addr.broadcast and addr.address and addr.address != addr.broadcast:  # AF_INET is for IPv4
                    ip_to_broadcast[addr.broadcast] = addr.address

        if ip_to_broadcast:
            return ip_to_broadcast

    ip_to_broadcast['255.255.255.255'] = getmyIP()
    return ip_to_broadcast