search

Avnu / detd

Proof-of-concept for a developer-friendly system service to handle time-sensitive applications.
BSD 3-Clause "New" or "Revised" License
17 stars 6 forks source link
detd developer-experience time-sensitive time-sensitive-networking tsn

readme

Time Sensitive System Service Prototype (detd)

A proof-of-concept for a developer-friendly system service to handle time-sensitive applications.

Overview

In a nutshell

This is a prototype of a system service to handle time-sensitive applications

Principles

Current functionality

The following snippet illustrates the add_talker interface:

interface_name = "eth0"
interval = 2 * 1000 * 1000 # ns for 2 ms
size = 1522                 # Bytes

txoffset = 250 * 1000       # ns for 250 us
addr = "7a:b9:ed:d6:d2:12"
vid = 3
pcp = 6

interface = Interface(interface_name)
traffic = TrafficSpecification(interval, size)
stream = StreamConfiguration(addr, vid, pcp, txoffset)

config = Configuration(interface, stream, traffic)

manager = Manager()
manager.add_talker(config)

In a little bit more detail:

  1. Takes stream configuration and traffic specification as an input
    • Interface to use
    • Stream configuration: txoffset, DMAC, VID,PCP
    • Traffic specification: Interval in nanoseconds, Bytes to transmit
  2. Detects and loads the specific network device handler based on the interface name
  3. Integrates all the inputs and generates the gating schedule
    • It is able to generate very complex taprio configurations with just the stream and traffic information
  4. Generates the commands implementing that configuration
    • taprio offload mode supported
  5. Applies the configuration by running the commands
    • When one of the commands fail, it is able to roll back the previous one to leave the system in a well-known state.
  6. Provides the stream handlers to the caller application
    • The basic implementation provides the VLAN interface name (e.g. eth0.3) and the socket prio to use to the calling application
    • The plan is to return a fully configured socket. The included client-server example is already implementing part of that flow.

Currently supported devices

Current limitations

See Contributing :)

Examples

Setup a talker from python

The following code will connect to the detd service and configure the specified stream.

from detd import *

def setup_stream_config():

    interface_name = "eno1"
    interval = 20 * 1000 * 1000 # ns
    size = 1522                 # Bytes

    txoffset = 250 * 1000       # ns
    addr = "03:C0:FF:EE:FF:4E"
    vid = 3
    pcp = 6
    interface = Interface(interface_name)
    stream = StreamConfiguration(addr, vid, pcp, txoffset)
    traffic = TrafficSpecification(interval, size)

    config = Configuration(interface, stream, traffic)

    return config

proxy = ServiceProxy()

config = setup_stream_config()
response = proxy.add_talker(config)

print(response)

This example adds the talker and then prints the VLAN configured interface and socket priority that the calling application should use to send data through that stream:

('eno1.3', 7)

The behaviour can be inspected in the log file /var/log/detd.log:

[2022-09-15 18:18:29,608 -     INFO]    detd.service             __init__() -  * * * detd Service starting * * *
[2022-09-15 18:18:29,608 -     INFO]    detd.service             __init__() - Initializing Service
[2022-09-15 18:18:29,609 -     INFO]    detd.manager             __init__() - Initializing Manager
[2022-09-15 18:18:29,609 -     INFO]    detd.service                  run() - Entering Service main loop
[2022-09-15 18:18:34,173 -     INFO]    detd.service                setup() - ============================== REQUEST DISPATCHED ==================================
[2022-09-15 18:18:34,173 -     INFO]    detd.service                setup() - Setting up ServiceRequestHandler
[2022-09-15 18:18:34,173 -     INFO]    detd.service               handle() - Handling request
[2022-09-15 18:18:34,178 -     INFO]    detd.devices             __init__() - Initializing IntelMgbeEhl
[2022-09-15 18:18:34,178 -     INFO]    detd.manager           add_talker() - Adding talker to Manager
[2022-09-15 18:18:34,178 -     INFO]    detd.manager             __init__() - Initializing InterfaceManager
[2022-09-15 18:18:34,178 -     INFO]    detd.mapping             __init__() - Initializing Mapping
[2022-09-15 18:18:34,179 -     INFO] detd.systemconf             __init__() - Initializing SystemConfigurator
[2022-09-15 18:18:34,179 -     INFO]  detd.scheduler             __init__() - Initializing Scheduler
[2022-09-15 18:18:34,179 -     INFO]    detd.manager           add_talker() - Adding talker to InterfaceManager
[2022-09-15 18:18:34,179 -     INFO]    detd.mapping       assign_and_map() - Assigning and mapping resources
[2022-09-15 18:18:34,179 -     INFO]  detd.scheduler                  add() - Adding traffic to schedule
[2022-09-15 18:18:34,179 -     INFO] detd.systemconf                setup() - Setting up platform and devices

detd code uses high level constructs. Only in the final step it translates to specific command calls. For the example above, such commands would be:

tc qdisc replace
         dev       eno1
         parent    root
         taprio
         num_tc    2
         map       0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
         queues    1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7
         base-time <now + 2 cycles>
         sched-entry S 01 250000
         sched-entry S 02 12176
         sched-entry S 01 19737824
         flags     0x2

ip link add
        link     eno1
        name     eno1.3
        type     vlan
        protocol 802.1Q
        id       3
        egress   0:0 1:1 2:2 3:3 4:4 5:5 6:7 7:7

ethtool --set-eee eth0 eee off

Setup a talker from python with options

The following code will connect to the detd service and configure the specified stream with options.

from detd import *

def setup_stream_config():

    interface_name = "eno1"
    interval = 20 * 1000 * 1000 # ns
    size = 1522                 # Bytes

    txoffset = 250 * 1000       # ns
    addr = "03:C0:FF:EE:FF:4E"
    vid = 3
    pcp = 6

    preemption = False
    launch_time_control = False
    tx_selection_offload = False
    datapath = DataPath.AF_PACKET
    tx_selection = TxSelection.EST

    interface = Interface(interface_name)
    stream = StreamConfiguration(addr, vid, pcp, txoffset)
    traffic = TrafficSpecification(interval, size)
    hints = Hints(tx_selection, tx_selection_offload, datapath, preemption, launch_time_control)

    config = Configuration(interface, stream, traffic, hints)

    return config

proxy = ServiceProxy()

config = setup_stream_config()
response = proxy.add_talker(config)

print(response)

This example specifies the map parameter for taprio, otherwise it behaves same as the example above.

Setup a talker stream for an arbitrary command, using a script that calls detd functions

The script setup_qos.sh allows for quick experimentation without modifying an existing application.

It performs the configuration using detd commands, as in the first example.

Then it uses the cgroup net_prio to automatically map the egress traffic from the command to a given socket prio. So all the queuing disciplines and PCP mapping in place works properly.

Example:

# 2ms period, 1522 bytes, TxOffset 250 microseconds, interface eth0
# stream DMAC AB:CD:EF:FE:DC:BA, stream VID 3, stream PCP 6
# Command: ping 8.8.8.8 for one second (Busybox's ping)

./setup_qos.sh --period 2000000 --bytes 1522 --offset 250000 --interface eth0 \
               --address AB:CD:EF:FE:DC:BA --vid 3 --pcp 6 \
               -- ping -w 1 8.8.8.8

Setup a talker stream using detd functions from python

Although it is not the intended use case, you could also call the detd classes from ad-hoc applications. Please note that in this case there is no connection to any service.

interface_name = "eth0"
interval = 20 * 1000 * 1000 # ns for 20 ms
size = 1522                 # Bytes

txoffset = 250 * 1000       # ns for 250 us
addr = "7a:b9:ed:d6:d2:12"
vid = 3
pcp = 6

interface = Interface(interface_name)
traffic = TrafficSpecification(interval, size)
stream = StreamConfiguration(addr, vid, pcp, txoffset)

config = Configuration(interface, stream, traffic)

manager = Manager()
manager.add_talker(config)

The only relevant change in the code above is using the class Manager instead of ServiceProxy to add the talker.

Regression testing

Test environments

By default, calls to system commands are mocked, so almost every code path can be exercised without TSN target hardware.

The test environment is controlled by the following environment variable.

DETD_TESTENV

Host testing is configured by default inside the test suite.

To override the default and setup target testing (e.g. calling actual commands):

DETD_TESTENV=TARGET

For integration testing, the service is spawned as a different process. This prevents regular use of patch. Hence, the Server class accepts a parameter test_mode, that when set to True will patch the required methods in order to maximize coverage. This is automatically performed when running the unittests.

Test the example "Setup a talker stream using detd functions from python"

Run the complete test suite

Integration tests include a client-server example in python.

Run all unit and integration tests in the development host test environment:

python3 -m unittest discover

A convenience script test.sh is included in the detd directory, that basically runs the above command:

cd detd
./test.sh

Getting involved

Installation

deb package

A convenience script package_debian.sh is provided to generate a deb file for easier installation on Debian-based distributions. It generates a deb file and stores it in /tmp.

cd tools
./package_debian.sh
apt install /tmp/detd_*deb

Upon installation, the service is started by systemd, with logging redirected to /var/log/detd.log:

tail /var/log/detd.log
[2022-09-14 23:02:28,780 -     INFO]    detd.service             __init__() -  * * * detd Service starting * * *
[2022-09-14 23:02:28,780 -     INFO]    detd.service             __init__() - Initializing Service
[2022-09-14 23:02:28,780 -     INFO]    detd.manager             __init__() - Initializing Manager
[2022-09-14 23:02:28,780 -     INFO]    detd.service                  run() - Entering Service main loop

At this point the service is ready to receive requests.

Docker

The script detd_docker_build.sh allows to build the deb packages inside a docker container in a convenient way. E.g. just run:

cd tools
./detd_docker_build.sh

Different variants are supported, like Debian Bookworm or Ubuntu 22.04. In order to build for a specific target, provide the name as argument. The value should match that used by the FROM clause inside a Dockerfile. For example:

./detd_docker_build.sh ubuntu:24.04

If you are behind a proxy and experience connectivity issues, e.g. to update and install the dependencies, you may want to include the following in the Dockerfile:

ENV http_proxy=http://your.http.proxy:port
ENV https_proxy=http://your.https.proxy:port

pip

The release process and security checks for the current release were conducted over the following specific versions:

To install:

git clone https://github.com/Avnu/detd.git
cd detd
python3 setup.py bdist_wheel
# Uninstall in case a previous version is there
# We install detd system wide
sudo pip3 uninstall detd
sudo pip3 install dist/detd-*.whl
sudo pip3 show detd

Optionally, you may want to run the test suite on host to make sure everything is in place:

python3 -m unittest discover
.....................................ss
----------------------------------------------------------------------
Ran 39 tests in 1.458s

OK (skipped=2)

Contributing

Junior tasks

Other improvements