search

zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.45k stars 6.4k forks source link

net: config: handle autoconfiguration of multiple interfaces #29750

Open cfriedt opened 3 years ago

cfriedt commented 3 years ago

Introduction

This is a proposal for a minimal subset of Devicetree bindings to allow for autoconfiguration of multiple network interfaces via DT_INST_FOREACH_STATUS_OKAY().

The primary reason for using DT is to allow for relatively easy scaling in comparison to Kconfig. The network stack was added to Zephyr before Devicetree came to Zephyr, when most devices had (at most!) 1 network interface. Now, we run on devices with multiple Ethernet ports, WiFi, BLE, etc. Imagine trying to use Kconfig to configure a 32-port router or something along those lines.

Problem description

Zephyr is beginning to see some increased need to support multiple network interfaces on a single device. Obvious examples would include dual-radio devices (with 5 GHz and 2.4 GHz transceivers) and dual-mac devices (multiplexing 2.4 GHz IEEE 802.15.4 and BLE).

We also need to support Ethernet, IP over CAN, SLIP, PPP, USB CDC ACM, among others.

The current issue is that our network stack only supports auto-configuration of a single network interface (and additionally, one IPv6 or IPv4 address per interface).

E.g. from samples/net/sockets/echo_server/prj.conf

CONFIG_NET_CONFIG_SETTINGS=y
CONFIG_NET_CONFIG_NEED_IPV6=y
CONFIG_NET_CONFIG_MY_IPV6_ADDR="2001:db8::1"
CONFIG_NET_CONFIG_PEER_IPV6_ADDR="2001:db8::2"
CONFIG_NET_CONFIG_NEED_IPV4=y
CONFIG_NET_CONFIG_MY_IPV4_ADDR="192.0.2.1"
CONFIG_NET_CONFIG_PEER_IPV4_ADDR="192.0.2.2"

Proposed change

Detailed RFC

Proposed change (Detailed)

How to associate IP addresses with a particular device? Either

For X in addr, netmask, and route, dns:

Either of the following (should debate the relative usefulness and choose 1):

Either of the following (should debate the relative usefulness and choose 1):

Address mode selection (an enumeration):

There are some special-case network interface types that might require additional DT bindings - e.g. PPP requires a peer address, IEEE 802.15.4 has short and long addresses, it may be desirable to specify an alternate MAC address for Ethernet, some configurations may require multicast addresses, etc. In the future, we may add TAP adaptors for packet sniffing, or ethernet bridging, or virtual interfaces.

Specification should be handled via inheritance in DT bindings.

An interesting aspect of this storage-wise, is that the number of IP addresses per interface would be exactly the number that are configured in Devicetree - no more, no less, which is nice in terms of code / memory usage.

Dependencies

Devicetree bindings are additive. However, the network subsystem might have some instances that will need some refactoring in the case that Kconfig values are used directly.

List specific areas of the network subsystem here that will require attention:

Concerns and Unresolved Questions

Currently, there are many samples and tests that depend on Kconfig as the primary means of network interface configuration. Those samples and tests would need to be updated.

Additionally, many board definitions and products rely on Kconfig to configure network interfaces.

Alternatives

Intentionally blank

Is your enhancement proposal related to a problem? Please describe.

Yes!

This was conceptualized when working on #29598.

When more than 1 IEEE 802.15.4 address is present, the echo_client / echo_server applications misbehave.

*** Booting Zephyr OS build zephyr-v2.4.0-1206-g98490c598ed7  ***
[00:00:00.001,983] <wrn> net_if: You have 1 IPv6 net_if addresses but 2 network interfaces
[00:00:00.002,014] <wrn> net_if: Consider increasing CONFIG_NET_IF_MAX_IPV6_COUNT value.
[00:00:00.010,650] <err> net_config: Cannot setup IEEE 802.15.4 interface (-22)
[00:00:00.010,650] <inf> net_config: Initializing network
[00:00:00.010,650] <inf> net_config: Waiting interface 1 (0x20001438) to be up...

I tried the following which should allow more than 1 network interface

CONFIG_NET_IF_MAX_IPV6_COUNT=2

And appended -DCONFIG_NET_CONFIG_IEEE802154_DEV_NAME="\"IEEE802154_1\"" to indicate that network autoconfiguration should use the secondary IEEE 802.15.4 network interface, but then it simply hangs indefinitely until there is a timeout.

*** Booting Zephyr OS build zephyr-v2.4.0-1206-g98490c598ed7  ***
[00:00:00.010,681] <err> net_config: Cannot setup IEEE 802.15.4 interface (-22)
[00:00:00.010,681] <inf> net_config: Initializing network
[00:00:00.010,711] <inf> net_config: Waiting interface 1 (0x20001438) to be up...
[00:01:33.013,763] <err> net_config: Timeout while waiting network interface
[00:01:33.013,793] <err> net_config: Network initialization failed (-62)
[00:01:33.013,885] <inf> net_echo_client_sample: Run echo client
[00:01:33.013,916] <inf> net_echo_client_sample: Network disconnected
[00:01:33.013,977] <inf> net_echo_client_sample: Network disconnected

I think it's attempting to initialize "IEEE802154_0" but I'm not 100% certain.

The particular code in play here is:

subsys/net/lib/config/ieee802154_settings.c: int z_net_config_ieee802154_setup(void);
subsys/net/lib/config/init.c: static void setup_ipv6(struct net_if *iface, uint32_t flags);

Describe the solution you'd like I think rather than "hard-coding" the network interfaces in Kconfig, we should probably introduce DT bindings for network configuration. That way, any number of interfaces can be instantiated and their IP addresses can be set in .dts rather than in a .conf file.

The DT bindings would need to cover a number of uses cases:

Describe alternatives you've considered See above

Additional context This issue was discovered in the process of #29598. We would ideally like to demonstrate IEEE 802.15.4 at 2.4 GHz working simultaneously with Sub GHz IEEE 802.15.4. Technically speaking, I could set -DCONFIG_IEEE802154_CC13XX_CC26XX=n on the command line here when building with overlay-802154-subg.conf but if that were the case, I would most likely also want an overlay-802154-multi.conf where both the 2.4 GHz and SubGHz PHYs are active.

Suggestions welcome. I also don't like manually setting CONFIG_IEEE802154_CC13XX_CC26XX=n - maybe there is a generic way to disable 2.4 GHz IEEE 802.15.4? There seems to be only one option for CONFIG_IEEE802154=y, maybe it would be wise to add CONFIG_IEEE802154_SUB_GHZ=y as well.

jukkar commented 3 years ago

The subsys/net/lib/config was never designed to support multiple network interfaces. That is why for example echo-server Kconfig file has separate config options for the other network interfaces which are in use if VLAN is enabled. So in this case the configuration problem is pushed to the application as it has better knowledge of the desired functionality. It would be great if we could have a simple way to configure multiple network interfaces automatically.

cfriedt commented 3 years ago

It would be great if we could have a simple way to configure multiple network interfaces automatically.

It's surprising how many times "Yes" is the answer to the question "Can DT fix that?"

jukkar commented 3 years ago

It's surprising how many times "Yes" is the answer to the question "Can DT fix that?"

Yes, DT could probably be used here. As always, patches are welcome.

cfriedt commented 2 years ago

@jukkar @rlubos

Just had a great use case for this 😁 I lent out my wrt54gl (an original!) a long time ago, and I might finally see it returned soon. Would be a fun retro-computing project to Zephyr running on it and work on getting all features supported. Might provide some interesting new features for Zephyr along the way.

cfriedt commented 2 years ago

@gmarull - PTAL when you have a moment, would be great to get your input here.

mbolivar-nordic commented 2 years ago

I generally agree that the right place for this in Zephyr is devicetree. This has always been our escape mechanism when Kconfig doesn't scale. I don't have enough networking knowledge to comment in depth, but I'd really love to see the networking subsystem become more DT aware in general. I do have some comments on the detailed RFC:

ipv4-X = (less-than) ip1 ip2 ip3 ip4 (greater-than); (array of uint32's, can be validated at compile-time)

I think you meant something like ipv4-X = <ip1 ip2 ip3 ip4>;, right? I'm not sure why you typed out "(less than)" and "(greater than)", haha.

If so, note that helper macros are always possible, and you can use a slightly different syntax, to get a bit more readable (but completely equivalent) results. Example:

ipv4-addr = <IPV4_ADDR(192, 168, 0, 1)>,
                   <IPV4_ADDR(...)>,
                   ...;

ipv6-X = [ip1], [ip2], [ip3]; (list of byte-arrays, does this work in DT?,

It will all just be concatenated into a single byte array.

Devicetree bindings are additive.

... by default, but you can do this too to just take exactly what you want from an include:

include:
  - name: foo.yaml
    property-allowlist:
      - i-want-this-one
      - and-this-one
  - name: bar.yaml
    property-blocklist:
      - do-not-include-this-one
      - or-this-one
cfriedt commented 2 years ago

@mbolivar-nordic - thanks for the suggestions (esp the ipv6 version). Had the same idea in mind with the IPv4 addresses.

Was just kind of rushing, talking, taking notes while I noticed <> did not render.

I like the idea of using binary versions of IP addresses but am also worried that they might be annoying for users.

It's not a problem for IPv4, but kind of annoying for IPv6 (unless we had some similar macro).

strings would generally make things a bit bloated, otoh.

@rlubos, do you know of a convenient macro for specifying IPv6 addresses in binary? I can imagine one way to do it, but it would involve some level of macrobatocs (h/t @henrikbrixandersen)

rlubos commented 2 years ago

@rlubos, do you know of a convenient macro for specifying IPv6 addresses in binary? I can imagine one way to do it, but it would involve some level of macrobatocs (h/t @henrikbrixandersen)

Nothing comes to my mind, unfortunately. Personally, I'd lean towards string representation of IP addresses, especially for IPv6, a commonly used test address 2001:db8::1 for example, the string representation is pretty neat, but imagine putting all those zeroes if we wanted to represent this in binary...

cfriedt commented 2 years ago

@rlubos, do you know of a convenient macro for specifying IPv6 addresses in binary? I can imagine one way to do it, but it would involve some level of macrobatocs (h/t @henrikbrixandersen)

Nothing comes to my mind, unfortunately. Personally, I'd lean towards string representation of IP addresses, especially for IPv6, a commonly used test address 2001:db8::1 for example, the string representation is pretty neat, but imagine putting all those zeroes if we wanted to represent this in binary...

Exactly - I occasional remind myself that this could probably be done at compile time via C++ and constexpr. In fact, I think that the majority of our ELF hacks could be done this way too, which would simplify the macOS port (cc @galak)

jadonk commented 1 year ago

Is there at least a way to dynamically switch between the multiple interfaces if I cannot get them both enabled at the same time? Having different software builds for this seems unreasonable.

rlubos commented 1 year ago

Is there at least a way to dynamically switch between the multiple interfaces if I cannot get them both enabled at the same time? Having different software builds for this seems unreasonable.

I'm not sure I follow, you can have mutliple interfaces enabled at the same time, the issue is about the net_config lib not being able to configure multiple interfaces at boot. But you still can configure them manually with net_if_* APIs if needed.

The IP stack will choose the right interface based on the destination IP address, or you can also use the SO_BINDTODEVICE socket option to bind a socket to a specific interface.

fabiobaltieri commented 1 year ago

Notes from the chat:

mkschreder commented 1 year ago

What about extending settings subsystem with features necessary for storing hierarchical network config data and then providing a way to compile in default settings based on yaml definitions?

jukkar commented 11 months ago

Hi all, It seems that we cannot use device tree for storing this configuration, and yaml has been mentioned in various places as a way to express the configuration data. So I was thinking this network subsystem configuration problem and came to this that the user could create yaml file that would describe network configuration data, then that data could be preprocessed to a settings db (and saved to a partition) which could then be loaded at runtime by a network config module which could then apply the configuration. The said configuration could be saved and then re-loaded later in the next boot.

Below is a simple example yaml file that illustrates how the config data could be used.

# Example of one interface selected by its name
- net_if: &main-interface
    name: eth0
    set-name: my-eth0
    link-address: 01:02:03:04:05:06
    MTU: 1500
    set-default: true
    IPv6:
      status: enabled
      addresses:
        - address: 2001:db8:110/64
      multicast-addresses:
        - address: ff05::114/16
        - address: ff15::115/16
      prefixes:
        - address: 2001:db8::/64
          lifetime: 1024
    IPv4:
      status: enabled
      addresses:
        - address: 192.0.2.10/24
      multicast-addresses:
        - address: 234.0.0.10/8
      gateway: 192.0.2.1
    DHCPv4: enabled
    DHCPv6: disabled
    IPv4-autoconf: disabled

# Example of another interface selected by its device
- net_if:
    device: ETH_DEVICE
    set-name: my-eth1
    flags: no-auto-start
    IPv4:
      status: enabled
      addresses:
        - address: 192.168.1.2/24
      multicast-addresses:
        - address: 234.0.0.10/8
      gateway: 192.168.110.1
    DHCPv4: enabled

# Example of virtual interface tied to the first one
- net_if:
    name: virt0
    set-name: virt0
    bind-to: *main-interface
    IPv6:
      status: enabled
      addresses:
        - address: 2001:db8:110/64
    IPv4:
      status: disabled

# Example of VLAN interface that attaches to eth0
- net_if: &vlan-interface
    set-name: vlan0
    bind-to: *main-interface
    VLAN:
      status: enabled
      tag: 1234
    DHCPv4: enabled

# IEEE 802.15.4 configuration data
- ieee-802.15.4:
    pan-id: 0xabcd
    channel: 26
    tx-power: 1
    security-key: foobar
    security-key-mode: 0
    security-level: 1
    ack-required: true

- sntp:
    server: sntp.foo.bar
    timeout: 30
    bind-to: *vlan-interface

WDYT?

cfriedt commented 11 months ago

@jukkar - thanks so much for being the first one to take the plunge 😁 At first glance, this looks fine to me, but a lot of things look fine at first glance.

And of course, in terms of the network elements you captured here, I think they make sense.

YAML makes complete sense from a high level - it is the glue that binds a lot of things together, and is supported quite well in many places. At the same time, I wonder if there is an actual standard schema of this sort (i.e. an IEEE spec or standard, or IETF RFC that captures best practices).

Food for thought:

The OpenWrt solution is (maybe not coincidentally?) like Zephyr's Settings - a key-value format (with extra "quirks") that is necessarily string based. Also, OpenWrt makes heavy use of Lua (which is actually pretty good for embedded DSLs and also structured data).

Devicetree

There is a YAML to DT converter maintained by friends at Konsulko. Yes, I know it sounds funny, but in all seriousness, a lot of markup languages can usually be translated from one format to another. With that, at least, we would be able to re-use some existing DT mechanisms where we get the compile-time const aspect of config data that (IMO) really serves Zephyr quite well.

So it's a +1 for the secondary SW configuration root node proposed by @carlocaione, but at the same time, it opens up a lot of other possibilities if combined with the YAML approach proposed by @jukkar.

At the same time, we don't yet incorporate any runtime DT (DTB) or YAML parsers, and storage is a bit of a concern.

Settings DB

I have concerns about the Settings DB. It is a key-value store and is necessarily string-based. Sure, it's human-readable, but so is JSON or YAML, or XML and those are standardized.

Concerns:

jukkar commented 11 months ago

Devicetree

There is a YAML to DT converter maintained by friends at Konsulko. Yes, I know it sounds funny, but in all seriousness, a lot of markup languages can usually be translated from one format to another. With that, at least, we would be able to re-use some existing DT mechanisms where we get the compile-time const aspect of config data that (IMO) really serves Zephyr quite well.

That is an interesting idea to use DT as an intermediate layer that passes the configuration data to zephyr instance. I just wonder the tooling to get this work during the build.

So it's a +1 for the secondary SW configuration root node proposed by @carlocaione, but at the same time, it opens up a lot of other possibilities if combined with the YAML approach proposed by @jukkar.

Indeed :+1:

At the same time, we don't yet incorporate any runtime DT (DTB) or YAML parsers, and storage is a bit of a concern.

I think we do not want to incorporate any new parsers to zephyr image. IMHO all the processing of yaml should be done in host side.

Settings DB

I have concerns about the Settings DB. It is a key-value store and is necessarily string-based. Sure, it's human-readable, but so is JSON or YAML, or XML and those are standardized.

I was mainly thinking Settings DB here as that could be used as a provision db inside zephyr. So the initial settings db could be provided by a yaml file, then that could be mounted in zephyr as read-write and any modifications to it could be remembered and restored in next boot. But that is kind of system level thing and I am not sure if we want to go there here and perhaps that is best left for the vendors to tackle. Bluetooth subsystem already has provision data using settings db, it is just not created / populated beforehand but during when device runs.

jukkar commented 11 months ago

I did some experimenting with yaml2dts which can be found at https://github.com/pantoniou/yamldt with these results:

Example: This is valid yaml code:

net:
  network-interfaces:
    # Example of one interface selected by its name
    - net-if: &main-interface
      name: eth0
      set-name: my-eth0
      set-default: true
      DHCPv4: enabled
      DHCPv6: disabled
      IPv4-autoconf: disabled
    # Example of second interface
    - net-if:
      name: eth1
      set-name: my-eth1
      DHCPv4: enabled
      IPv4-autoconf: disabled

Which when "pretty-printed" produces this output

$ ./pretty-print.sh < eth0-valid.yaml
{'net': {'network-interfaces': [{'DHCPv4': 'enabled',
                                 'DHCPv6': 'disabled',
                                 'IPv4-autoconf': 'disabled',
                                 'name': 'eth0',
                                 'net-if': None,
                                 'set-default': True,
                                 'set-name': 'my-eth0'},
                                {'DHCPv4': 'enabled',
                                 'IPv4-autoconf': 'disabled',
                                 'name': 'eth1',
                                 'net-if': None,
                                 'set-name': 'my-eth1'}]}}

I had to change this to:

net:
  network-interfaces:
    # Example of one interface selected by its name
    net-if@a: &main-interface
      name: eth0
      set-name: my-eth0
      set-default: true
      DHCPv4: enabled
      DHCPv6: disabled
      IPv4-autoconf: disabled
    # Example of second interface
    net-if@b:
      name: eth1
      set-name: my-eth1
      DHCPv4: enabled
      IPv4-autoconf: disabled

which "pretty-printed" looks like this

$ ./pretty-print.sh < eth0.yaml
{'net': {'network-interfaces': {'net-if@a': {'DHCPv4': 'enabled',
                                             'DHCPv6': 'disabled',
                                             'IPv4-autoconf': 'disabled',
                                             'name': 'eth0',
                                             'set-default': True,
                                             'set-name': 'my-eth0'},
                                'net-if@b': {'DHCPv4': 'enabled',
                                             'IPv4-autoconf': 'disabled',
                                             'name': 'eth1',
                                             'set-name': 'my-eth1'}}}}

which was then accepted by the tool yamldt eth0.yaml -o eth0.dts and it generated this output

/dts-v1/;
/ {
    net {
        network-interfaces {
            main-interface: net-if@a {
                name = "eth0";
                set-name = "my-eth0";
                set-default;
                DHCPv4 = "enabled";
                DHCPv6 = "disabled";
                IPv4-autoconf = "disabled";
            };
            net-if@b {
                name = "eth1";
                set-name = "my-eth1";
                DHCPv4 = "enabled";
                IPv4-autoconf = "disabled";
            };
        };
    };
};
cfriedt commented 11 months ago

I did some experimenting with yaml2dts which can be found at https://github.com/pantoniou/yamldt with these results:

  • The tool seems to be abandoned, last update is from 2017.

Hard to say. I would guess that it is in use in Yocto-related things. The author would likely accept PRs and has also actively contributed todtc for a long time.

https://github.com/pantoniou/dtc

The yaml2dts output looks a little unexpected. Would be good to ensure that we get the preferred / correct syntax.

cc @pantoniou

carlescufi commented 9 months ago

What about extending settings subsystem with features necessary for storing hierarchical network config data and then providing a way to compile in default settings based on yaml definitions?

The settings subsystem is entirely optional, and adding a dependency to it is probably not a good idea. Also, this would then need pre-baking the settings which would require yet more tooling.