search

eclipse-cyclonedds / cyclonedds

Eclipse Cyclone DDS project
https://projects.eclipse.org/projects/iot.cyclonedds
Other
889 stars 363 forks source link
dds

readme

GitHub release Build Status Coverity Status Coverage License License Website Community

Eclipse Cyclone DDS

Eclipse Cyclone DDS is a very performant and robust open-source implementation of the OMG DDS specification. Cyclone DDS is developed completely in the open as an Eclipse IoT project (see eclipse-cyclone-dds) with a growing list of adopters (if you're one of them, please add your logo). It is a tier-1 middleware for the Robot Operating System ROS 2.

What is DDS?

DDS is the best-kept secret in distributed systems, one that has been around for much longer than most publish-subscribe messaging systems and still outclasses so many of them. DDS is used in a wide variety of systems, including air-traffic control, jet engine testing, railway control, medical systems, naval command-and-control, smart greenhouses and much more. In short, it is well-established in aerospace and defense but no longer limited to that. And yet it is easy to use!

Types are usually defined in IDL and preprocessed with the IDL compiler included in Cyclone, but our Python binding allows you to define data types on the fly:

from dataclasses import dataclass
from cyclonedds.domain import DomainParticipant
from cyclonedds.core import Qos, Policy
from cyclonedds.pub import DataWriter
from cyclonedds.sub import DataReader
from cyclonedds.topic import Topic
from cyclonedds.idl import IdlStruct
from cyclonedds.idl.annotations import key
from time import sleep
import numpy as np
try:
    from names import get_full_name
    name = get_full_name()
except:
    import os
    name = f"{os.getpid()}"

# C, C++ require using IDL, Python doesn't
@dataclass
class Chatter(IdlStruct, typename="Chatter"):
    name: str
    key("name")
    message: str
    count: int

rng = np.random.default_rng()
dp = DomainParticipant()
tp = Topic(dp, "Hello", Chatter, qos=Qos(Policy.Reliability.Reliable(0)))
dw = DataWriter(dp, tp)
dr = DataReader(dp, tp)
count = 0
while True:
    sample = Chatter(name=name, message="Hello, World!", count=count)
    count = count + 1
    print("Writing ", sample)
    dw.write(sample)
    for sample in dr.take(10):
        print("Read ", sample)
    sleep(rng.exponential())

Today DDS is also popular in robotics and autonomous vehicles because those really depend on high-throughput, low-latency control systems without introducing a single point of failure by having a message broker in the middle. Indeed, it is by far the most used and the default middleware choice in ROS 2. It is used to transfer commands, sensor data and even video and point clouds between components.

The OMG DDS specifications cover everything one needs to build systems using publish-subscribe messaging. They define a structural type system that allows automatic endianness conversion and type checking between readers and writers. This type system also supports type evolution. The interoperable networking protocol and standard C++ API make it easy to build systems that integrate multiple DDS implementations. Zero-configuration discovery is also included in the standard and supported by all implementations.

DDS actually brings more: publish-subscribe messaging is a nice abstraction over "ordinary" networking, but plain publish-subscribe doesn't affect how one thinks about systems. A very powerful architecture that truly changes the perspective on distributed systems is that of the "shared data space", in itself an old idea, and really just a distributed database. Most shared data space designs have failed miserably in real-time control systems because they provided strong consistency guarantees and sacrificed too much performance and flexibility. The eventually consistent shared data space of DDS has been very successful in helping with building systems that need to satisfy many "ilities": dependability, maintainability, extensibility, upgradeability, ... Truth be told, that's why it was invented, and publish-subscribe messaging was simply an implementation technique.

Cyclone DDS aims at full coverage of the specs and today already covers most of this. With references to the individual OMG specifications, the following is available:

The network stack in Cyclone DDS has been around for over a decade in one form or another and has proven itself in many systems, including large, high-availability ones and systems where interoperatibility with other implementations was needed.

This repository provides the core of Cyclone DDS including its C API, the OMG C++ and the Python language bindings are in sibling repositories.

Consult the roadmap for a high-level overview of upcoming features.

Getting Started

Building Eclipse Cyclone DDS

In order to build Cyclone DDS you need a Linux, Mac or Windows 10 machine (or, with some caveats, a *BSD, QNX, OpenIndiana or a Solaris 2.6 one) with the following installed on your host:

If you want to play around with the parser you will need to install the bison parser generator. On Ubuntu apt install bison should do the trick for getting it installed. On Windows, installing chocolatey and choco install winflexbison3 should get you a long way. On macOS, brew install bison is easiest.

To obtain Eclipse Cyclone DDS, do

$ git clone https://github.com/eclipse-cyclonedds/cyclonedds.git
$ cd cyclonedds
$ mkdir build

Depending on whether you want to develop applications using Cyclone DDS or contribute to it you can follow different procedures:

Build configuration

There are some configuration options specified using CMake defines in addition to the standard options like CMAKE_BUILD_TYPE:

For application developers

To build and install the required libraries needed to develop your own applications using Cyclone DDS requires a few simple steps. There are some small differences between Linux and macOS on the one hand, and Windows on the other. For Linux or macOS:

$ cd build
$ cmake -DCMAKE_INSTALL_PREFIX=<install-location> ..
$ cmake --build .

and for Windows:

$ cd build
$ cmake -G "<generator-name>" -DCMAKE_INSTALL_PREFIX=<install-location> ..
$ cmake --build .

where you should replace <install-location> by the directory under which you would like to install Cyclone DDS and <generator-name> by one of the ways CMake generators offer for generating build files. For example, "Visual Studio 15 2017 Win64" would target a 64-bit build using Visual Studio 2017.

To install it after a successful build, do:

$ cmake --build . --target install

which will copy everything to:

Depending on the installation location you may need administrator privileges.

At this point you are ready to use Eclipse Cyclone DDS in your own projects.

Note that the default build type is a release build with debug information included (RelWithDebInfo), which is generally the most convenient type of build to use from applications because of a good mix between performance and still being able to debug things. If you'd rather have a Debug or pure Release build, set CMAKE_BUILD_TYPE accordingly.

Contributing to Eclipse Cyclone DDS

We very much welcome all contributions to the project, whether that is questions, examples, bug fixes, enhancements or improvements to the documentation, or anything else really. When considering contributing code, it might be good to know that build configurations for Azure pipelines are present in the repository and that there is a test suite built using a simple testing framework and CTest that can be built locally if desired. To build it, set the cmake variable BUILD_TESTING to on when configuring, e.g.:

$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=ON ..
$ cmake --build .
$ ctest

Documentation

The documentation is still rather limited and some parts of it are still only available in the form of text files in the docs directory. This README is usually out-of-date and the state of the documentation is slowly improving, so it definitely worth hopping over to have a look.

Building and Running the Roundtrip Example

We will show you how to build and run an example program that measures latency. The examples are built automatically when you build Cyclone DDS, so you don't need to follow these steps to be able to run the program, it is merely to illustrate the process.

$ mkdir roundtrip
$ cd roundtrip
$ cmake <install-location>/share/CycloneDDS/examples/roundtrip
$ cmake --build .

On one terminal start the application that will be responding to pings:

$ ./RoundtripPong

On another terminal, start the application that will be sending the pings:

$ ./RoundtripPing 0 0 0
# payloadSize: 0 | numSamples: 0 | timeOut: 0
# Waiting for startup jitter to stabilise
# Warm up complete.
# Latency measurements (in us)
#             Latency [us]                                   Write-access time [us]       Read-access time [us]
# Seconds     Count   median      min      99%      max      Count   median      min      Count   median      min
    1     28065       17       16       23       87      28065        8        6      28065        1        0
    2     28115       17       16       23       46      28115        8        6      28115        1        0
    3     28381       17       16       22       46      28381        8        6      28381        1        0
    4     27928       17       16       24      127      27928        8        6      27928        1        0
    5     28427       17       16       20       47      28427        8        6      28427        1        0
    6     27685       17       16       26       51      27685        8        6      27685        1        0
    7     28391       17       16       23       47      28391        8        6      28391        1        0
    8     27938       17       16       24       63      27938        8        6      27938        1        0
    9     28242       17       16       24      132      28242        8        6      28242        1        0
   10     28075       17       16       23       46      28075        8        6      28075        1        0

The numbers above were measured on Mac running a 4.2 GHz Intel Core i7 on December 12th 2018. From these numbers you can see how the roundtrip is very stable and the minimal latency is now down to 17 micro-seconds (used to be 25 micro-seconds) on this HW.

Performance

Reliable message throughput is over 1MS/s for very small samples and is roughly 90% of GbE with 100 byte samples, and latency is about 30us when measured using ddsperf between two Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz (that's 2012 hardware ...) running Ubuntu 16.04, with the executables built on Ubuntu 18.04 using gcc 7.4.0 for a default (i.e., "RelWithDebInfo") build.

ThroughputThroughput

This is with the subscriber in listener mode, using asynchronous delivery for the throughput test. The configuration is a marginally tweaked out-of-the-box configuration: an increased maximum message size and fragment size, and an increased high-water mark for the reliability window on the writer side. For details, see the scripts directory, the environment details and the throughput and latency data underlying the graphs. These also include CPU usage (throughput and latency) and memory usage.

Run-time configuration

The out-of-the-box configuration should usually be fine, but there are a great many options that can be tweaked by creating an XML file with the desired settings and defining the CYCLONEDDS_URI to point to it. E.g. (on Linux):

$ cat cyclonedds.xml
<?xml version="1.0" encoding="UTF-8" ?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
    <Domain Id="any">
        <General>
            <Interfaces>
                <NetworkInterface autodetermine="true" priority="default" multicast="default" />
            </Interfaces>
            <AllowMulticast>default</AllowMulticast>
            <MaxMessageSize>65500B</MaxMessageSize>
        </General>
        <Discovery>
            <EnableTopicDiscoveryEndpoints>true</EnableTopicDiscoveryEndpoints>
        </Discovery>
        <Internal>
            <Watermarks>
                <WhcHigh>500kB</WhcHigh>
            </Watermarks>
        </Internal>
        <Tracing>
            <Verbosity>config</Verbosity>
            <OutputFile>cdds.log.${CYCLONEDDS_PID}</OutputFile>
        </Tracing>
    </Domain>
</CycloneDDS>
$ export CYCLONEDDS_URI=file://$PWD/cyclonedds.xml

(on Windows, one would have to use set CYCLONEDDS_URI=file://... instead.)

This example shows a few things:

Background information on configuring Cyclone DDS can be found here and a list of settings is available.

Trademarks