search

ipilcher / fdf

Flexible Discovery Forwarder
GNU General Public License v2.0
11 stars 1 forks source link

readme

FDF - Flexible Discovery Forwarder

© 2022, 2023 Ian Pilcher <arequipeno@gmail.com>

Introduction

FDF is a highly configurable service that forwards broadcast and multicast discovery packets between networks.

Many network devices use some type of discovery protocol, which allows them to be automatically discovered by other devices or applications on the network. Examples include:

Most of these discovery protocols have been developed with the assumption that they will be used on a simple residential network with a single subnet on a single layer 2 domain (segment). They mostly use traffic types (IPv4 broadcast or local subnetwork multicast) that cannot be routed between networks. Even when a discovery protocol does use a routable multicast address (such as SSDP's 239.255.255.250), multicast routing capability is rare in residential routers, and it can be difficult to configure on those devices and operating systems that do offer support.

As IoT and home automation devices have proliferated, and consumers have become more conscious of privacy and security, more and more people want to separate their residential network into multiple sub-networks in order to segretate different device and traffic types and trust levels, and control which networks and devices are allowed to communicate with one another (and with external networks). Of course, this breaks these discovery protocols.

Fortunately, most network discovery protocols work just fine as long as the initial discovery message reaches the device or service to be discovered somehow. The "discoveree" typically does not verify that the discovery message originated on its local network; it simply sends a response directly to that message's source. If the network has been configured to route the response, it will be received by the "discoverer," and communication between the two will proceed normally (assuming that the network has been configured to route all of the required traffic).

FDF forwards broadcast and multicast discovery packets between networks, so discovery protocols designed for "flat" networks can work in more complex environments. FDF is not normally involved in routing unicast discovery responses; the network itself should be configured to route those packets. (But see the IP Set and nftables set filters.)

NOTE: The multicast DNS (mDNS) protocol does not follow the traffic pattern described above. mDNS queries and responses are both typically sent via IP multicast. Thus, both queries and responses must be forwared to enable multicast DNS across separate networks. See the Multicast DNS filter.

Building

Build Requirements

FDF has been developed and tested on Linux and GCC. Compatibility with other operating systems and compilers is unknown. (FDF does make use of several GCC extensions, as well as the Linux-specific epoll API, and the IP set and nftables set filters are Linux-specific.)

FDF requires three libraries — JSON-C and libmnl, which are both commonly available in Linux distribution package repositories, and libSAVL, which must be compiled and installed as documented here. The development packages or files for all three libraries must be installed in order to build FDF and all of the included filters.

NOTES:

  • libmnl is required only by the IP set and nftables set filters, which are not required.

  • If FDF will be installed on a system with SELinux enabled, see FDF SELinux Support for instructions to build, install, and configure the FDF SELinux policy module.

Compiling

Ensure that the required libraries and development files are installed, clone this repository, and change to its (src) directory. For example:

$ rpm -q json-c-devel libmnl-devel libsavl-devel
json-c-devel-0.15-2.fc35.x86_64
libmnl-devel-1.0.4-14.fc35.x86_64
libsavl-devel-0.7.1-1.fc35.x86_64

$ git clone https://github.com/ipilcher/fdf.git
Cloning into 'fdf'...
remote: Enumerating objects: 27, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 27 (delta 5), reused 23 (delta 3), pack-reused 0
Receiving objects: 100% (27/27), 32.43 KiB | 2.16 MiB/s, done.
Resolving deltas: 100% (5/5), done.

$ cd fdf/src

Ensure that the filter API version in fdf-filter.h is up to date.

$ ./apiver.sh

Build the daemon (fdfd).

$ gcc -std=gnu99 -O3 -Wall -Wextra -Wcast-align -o fdfd *.c -lsavl -ljson-c \
    -ldl -Wl,--dynamic-list=symlist

Build some or all of the included filters. For example:

$ cd filters

$ gcc -std=gnu99 -O3 -Wall -Wextra -Wcast-align -shared -fPIC -o mdns.so \
    -I.. mdns.c -lsavl

$ gcc -std=gnu99 -O3 -Wall -Wextra -Wcast-align -shared -fPIC -o ipset.so \
    -I.. ipset.c -lmnl

$ gcc -std=gnu99 -O3 -Wall -Wextra -Wcast-align -shared -fPIC -o nft-set.so \
    -I.. nft-set.c -lmnl

NOTE: The compiler options above provide maximum compatibility, across GCC versions.

-std=gnu99 is required only when using an older GCC version (such as GCC 4.8 on CentOS 7) that does not enable C99 features by default.

-Wcast-align can help to identify alignment problems on platforms that differentiate (at the instruction set level) between aligned and unaligned memory access. On these platforms, using aligned memory access instructions (which are preferred for performance reasons) with an incorrectly aligned address will cause a bus error, which will usually terminate the program. (Even worse, the processor may simply round the address down to a correctly aligned value, which will cause an incorrect memory location to be read or written.) Many of the RISC processors in residential routers behave in one of these ways.

x86 processors do not use different instructions for aligned and unaligned memory access (although use of unaligned addresses may affect performance or atomicity), so -Wcast-align has no effect when GCC is targeting an x86 platform. More recent versions of GCC support -Wcast-align=strict, which will cause GCC to issue alignment warnings even when it is targeting a platform that can tolerate unaligned memory access.

Configuration

FDF uses a JSON configuration file to control its operation. This configuration file must contain a single JSON object (dictionary), and the top-level object must contain 2 or 3 members. The matches and listen members are required; the filters member is optional.

NOTE: FDF does not perform schema validation of its configuration file. As a result, additional object members at any level are silently ignored (including members with misspelled names).

The skeleton of a configuration file that includes a filters member appears as follows.

{
    "filters": {

    },
    "matches": {

    },
    "listen": {

    }
}

Filters

The optional filters member of the configuration object specifies one or more dynamically loaded filter modules (shared objects). Filter modules can be used to pass or drop packets based on their payload, forward a packet to a specific network interface (see note in Listeners), or otherwise extend the functionality of the FDF daemon. (See the FDF Filter API.)

FDF currently includes three filter modules.

Each member of the filters object defines a filter instance of that name. Each filter instance must contain 1 or 2 members — file (required) and args (optional). file must be a JSON string that specifies the full path of the shared object to be loaded (unless the shared object is within the normal library search path). If present, args must be an array of JSON strings, which will be passed to the filter's initialization function to initialize the filter instance. (The name of the filter instance and the path to the shared object are also passed.)

The configuration fragment below creates two instances of the mDNS filter and one instance of the IP set filter.

    "filters": {
        "mdns_query": {
            "file": "./filters/mdns.so",
            "args": [ "mode=stateful", "forward=queries", "ipset=yes" ]
        },
        "mdns_response": {
            "file": "./filters/mdns.so",
            "args": [ "forward=responses" ]
        },
        "ipset_mdns": {
            "file": "./filters/ipset.so",
            "args": [ "set_name=MDNS_CLIENTS" ]
        }
    }

Matches

The (required) matches member of the configuration object defines address/port (or address/port/filters) tuples that identify network traffic. As with filter instances, the name of the JSON member determines the name of the match.

Each match must contain 2 or 3 members — addr (required), port (required), and filters (optional). addr must be a JSON string that contains an IPv4 address, in standard dotted decimal notation. The address must be the IPv4 broadcast address (255.255.255.255) or an IPv4 multicast address. port must be a JSON number (i.e. unquoted) that represents a valid UDP port (1 - 65535).

NOTE: FDF does not currently support IPv6. Very few of the devices that use these protocols support IPv6, and none of them are IPv6-only. (But see this issue.)

If present, filters must be an array of JSON strings, each of which is the name of a filter instance defined in the filters object. For each packet received on the match's specified address and port, the filter instances will be called in the order listed (unless a filter instance returns a value that prevents subsequent filters from being called); see below.

The configuration fragment below defines matches for several different types of traffic, using the filter instances shown above.

    "matches": {
        "mdns_query": {
            "addr": "224.0.0.251",
            "port": 5353,
            "filters": [ "mdns_query", "ipset_mdns" ]
        },
        "mdns_response": {
            "addr": "224.0.0.251",
            "port": 5353,
            "filters": [ "mdns_response" ]
        },
        "ssdp": {
            "addr": "239.255.255.250",
            "port": 1900
        },
        "hdhomerun": {
            "addr": "255.255.255.255",
            "port": 65001
        }
    }

Filter Chaining

The mdns_query match above is defined with multiple filter instances, a configuration called a filter chain. When a packet is received by a listener that uses this match, the packet will be passed to each filter instance in the chain sequentially (unless a filter instance returns a result value that terminates filter processing of the packet). In this configuration, the packet will first be passed to the mdns_query filter instance. Depending on the value returned by mdns_query, the packet may then be passed to the ipset_mdns filter instance.

The ultimate disposition of the packet (forwarded or dropped) is determined by the values returned by the filter instances in the chain. See Match Function for more information.

NOTE: It is not usually possible to arbitrarily chain filter modules. The modules being chained must specifically support such use. For example, the mdns_query filter instance above includes the ipset=yes argument. This causes the mDNS filter to behave in a way that is compatible with this configuration. (See IP Set Mode.)

Listeners

The (required) listen member of the configuration object specifies the network interfaces on which FDF will listen, the types of traffic (matches) for which it will listen on those interfaces, and the networks to which matching traffic will be forwarded.

Each member of the listen object identifies (by interface name) a network interface on which FDF will listen. Within that listen interface object, the name of each member identifies a match defined in the matches object, and the value of each member must be a list (array) of network interface names (JSON strings).

Each combination of a listen interface and a match defines a listener. Traffic that is received by a listener (and is not dropped due to a filter return value) will be forwarded to all of the network interfaces listed for that listener.

NOTE: As discussed above, it is possible for a filter to set a specific forward interface for a packet. That interface must be one of the forward interfaces listed for the listener that received the packet.

For example, assume that FDF is running on a system with 4 network interfaces.

Consider the following configuration fragment, which builds on the examples above.

    "listen": {
        "eth0": {
            "mdns_query": [ "eth1" ],
            "hdhomerun": [ "eth2" ]
        },
        "eth1": {
            "mdns_response": [ "eth0", "eth3" ],
            "ssdp": [ "eth3" ]
        },
        "eth3": {
            "mdns_query": [ "eth1" ]
        }
    }

This configuration has the following effects.

Validation

FDF uses the JSON-C library to parse its configuration. JSON-C has numerous benefits, but it does have some limitations.

The first two issues can be addressed by using a seperate JSON tool to validate the configuration and check for duplicate member names. jsonlint, part of the demjson Python module, is one such tool. A similar tool is available online at https://jsonlint.com/. The last issue could be addressed by with a JSON schema. (See this issue.)

Running FDF

Runtime Requirements

Running fdfd requires JSON-C, libSAVL, and (if using the IP set or nftables set filter) libmnl. The corresponding development files are not needed just to run the daemon.

It also has several network-related requirements.

Finally, the daemon must run either as the root user or with certain capabilities.

NOTE: If FDF is running on a system with SELinux enabled, see FDF SELinux Support.

Running fdfd

To run fdfd from a command prompt, create a configuration file and execute fdfd (usually as root) with any required command line options. fdfd accepts the following options.

NOTE: Both -d and -p (or their longer equivalents) must be specified to enable logging of packet-specific DEBUG level messages from filters.

fdfd can also be run as a systemd service, using the unit file in this repository (fdfd.service). To use the unit file unchanged, perform the following steps as the root user.

For example (from the top-level directory of the repository):

# cp systemd/fdfd.service /etc/systemd/sytem/

# cp src/fdfd /usr/local/bin/

# mkdir /usr/local/lib64/fdf-filters

# cp src/filters/*.so /usr/local/lib64/fdf-filters/

# cp fdf-config.json /etc/

# systemctl daemon-reload

# systemctl enable fdfd --now
Created symlink from /etc/systemd/system/multi-user.target.wants/fdfd.service to /etc/systemd/system/fdfd.service.

# systemctl status fdfd
● fdfd.service - Flexible Discovery Forwarder daemon
   Loaded: loaded (/etc/systemd/system/fdfd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2022-02-26 17:33:49 CST; 36s ago
 Main PID: 18317 (fdfd)
   CGroup: /system.slice/fdfd.service
           └─18317 /usr/local/bin/fdfd

Feb 26 17:33:49 asterisk.penurio.us systemd[1]: Started Flexible Discovery Forwarder daemon.
Feb 26 17:33:49 asterisk.penurio.us fdfd[18317]: INFO: filter.c:214: Loaded filter (mdns_query..." ]
Feb 26 17:33:49 asterisk.penurio.us fdfd[18317]: INFO: filter.c:214: Loaded filter (mdns_answe..." ]
Hint: Some lines were ellipsized, use -l to show in full.