// Licensed to the Apache Software Foundation (ASF) under one // or more contributor license agreements. See the NOTICE file // distributed with this work for additional information // regarding copyright ownership. The ASF licenses this file // to you under the Apache License, Version 2.0 (the // "License"); you may not use this file except in compliance // with the License. You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, // software distributed under the License is distributed on an // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY // KIND, either express or implied. See the License for the // specific language governing permissions and limitations // under the License. :toc: :toclevels: 5 = Skupper Router

image:https://github.com/skupperproject/skupper-router/actions/workflows/build.yaml/badge.svg[ "GitHub Actions" link="https://github.com/skupperproject/skupper-router/actions/workflows/build.yaml"] image:https://app.travis-ci.com/skupperproject/skupper-router.svg?branch=main[ "Travis CI", link="https://app.travis-ci.com/skupperproject/skupper-router"] image:https://codecov.io/github/skupperproject/skupper-router/branch/main/graph/badge.svg?token=XGZ7WM8SYU[ "codecov", link="https://app.codecov.io/github/skupperproject/skupper-router"] image:https://img.shields.io/github/license/skupperproject/skupper-router.svg[ "License", link="https://github.com/skupperproject/skupper-router/blob/main/LICENSE"]

Skupper Router is a high-performance, lightweight AMQP 1.0 message router. It provides flexible and scalable interconnect between any AMQP endpoints, and is forked from the https://github.com/apache/qpid-dispatch/[qpid-dispatch] project to focus on the https://skupper.io[Skupper] use case.

== Building and testing

NOTE: Skupper router only supports the Linux OS.

=== Dependencies

To build skupper router on a yum-based Linux system, you need the following packages installed:

• qpid-proton-c-devel
• python3-qpid-proton
• cmake
• make
• gcc
• gcc-c++
• python3-devel
• cyrus-sasl-plain
• cyrus-sasl-devel
• libnghttp2-devel
• asciidoc (for building docs)
• asciidoctor (for building docs)

To build formatted documentation (man pages, HTML, PDF) see the requirements in doc/README.adoc

=== Build

From the skupper-router directory:

[source,shell script]

$ mkdir my_build # or directory of your choice. $ cd my_build $ cmake .. $ make

=== Running The Tests

From the <build> directory you can run all the system- and unit-tests with: [source,shell script]

$ ctest -VV

The ctest tool uses the script <build>/test/run.py to set up the correct environment for tests. You can use it to run tests individually from the <build>/tests directory.

.Example [source,shell script]

$ ./run.py unit_tests_size 3 $ ./run.py -m unittest system_tests_qdstat

Run it without arguments to get a summary of how it can be used: [source,shell script]

$ ./run.py

=== Test-only dependencies

.Install package dependencies (Fedora) [source, shell script]

dnf install curl nmap-ncat

.Install Python test dependencies from requirements-dev.txt [source, shell script]

python3 -m pip install --user --upgrade pip python3 -m pip install --user -r ./requirements-dev.txt

== Development

=== git pre-commit hook

This project uses the https://github.com/pre-commit/pre-commit[pre-commit] git hook management framework.

.Install and activate pre-commit (Fedora) [source, shell script]

$ dnf install pre-commit $ pre-commit install

The second command will install the pre-commit script into your ./.git/hooks/pre-commit.

Hooks are configured in .pre-commit-config.yaml. The git clang-format hook enforces xref:docs/notes/code-conventions.md[code-conventions] for modified C code upon git commit. Every time a user attempts a commit, clang-format is run for the staged files and if an incorrectly formatted line is detected, the commit aborts. The changed lines are automatically formatted during this operation, so after reviewing the changes, the user can then repeat the commit.

Read this https://www.redhat.com/sysadmin/git-hooks[git hooks article] on redhat.com (as well as documentation for pre-commit) to learn more.

=== Description of the individual test-only dependencies

The HTTP2 system tests (tests/system_tests_http2.py) use the Python Quart and hyper-h2 frameworks to start a HTTP2 server. The HTTP2 system tests only runs if

1. Python version >= 3.7
2. Python Web Microframework Quart version >= 0.13
3. curl is available
4. hyper-h2 is available (pure-Python implementation of a HTTP/2 protocol stack)

The TCP system tests (tests/system_tests_tcp_adaptor.py) use the Python selectors module when running echo clients and servers. The TCP system tests run only if Python selectors is available. Several tests require the nc (ncat) binary to be present.

.Install curl and ncat (on Fedora) [source, shell script]

dnf install curl nmap-ncat

.Install quart, h2 and selectors [source,shell script]

pip3 install --user quart h2 selectors

Websocket system tests use the Python websockets asyncio module.

.Install websockets [source,shell script]

pip3 install --user websockets

The gRPC system tests (tests/system_tests_grpc.py) use grpcio and protobuf modules.

.Install gRPC libraries [source,shell script]

pip3 install --user grpcio protobuf

In order to regenerate the auto generated pb2 files used by system_tests_grpc.py, you must also install the following dependency:

[source,shell script]

pip3 install --user grpcio-tools

And run the following command to generate grpc code:

[source,shell script]

python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. ./friendship.proto

The system tests are implemented using Python's unittest library. This library is used to run the tests by default. The tests can be also run using xmlrunner or pytest. Pytest can generate a JUnit-compatible XML report containing an entry for each Python test method. After running the tests, all XML reports can be found under tests/junitxmls in your build directory:

[source,shell script]

cmake .. -DPYTHON_TEST_COMMAND='-m;pytest;-vs;--junit-xml=junitxmls/${py_test_module}.xml;--pyargs;${py_test_module}'

=== Runner for skrouterd in tests

System tests can be configured to run skrouterd processes with an arbitrary wrapper. To do this, set the QDROUTERD_RUNNER CMake option to a string that will be prepended before all skrouterd invocations during testing. The following example illustrates how to run the router under gdb, to obtain a backtrace if the router crashes.

[source,shell script]

cmake .. -DQDROUTERD_RUNNER="gdb -quiet -iex 'set pagination off' -iex 'set debuginfod enabled on' -ex run -ex 'thread apply all bt' -ex 'quit $_exitcode' --batch --args"

=== Test Suite Code Coverage (GNU tools only)

Use coverage analysis to ensure that all code paths are exercised by the test suite. To run the tests and perform code coverage analysis:

. Install the lcov package [source,shell script] $ yum install lcov

. Configure and build for the Coverage build type (from the directory): [source,shell script] $ cmake -DCMAKE_BUILD_TYPE=Coverage .. && make

. Run the test suite and generate the coverage html output [source,shell script] $ ctest && make coverage

. Use your browser to navigate to <build>/coverage_results/html/index.html

=== Clean build, install and test

WARNING: Any preexisting directories 'build' and 'install' are deleted.

Run the following command:

[source]

$ source config.sh; test.sh

This script then does the following:

• performs a fresh cmake and make in directory 'build'
• runs unit tests (not system tests) in 'build'
• performs 'make install' into the directory 'install'
• runs system tests on the installation in 'install'.

=== Run Time Validation

The CTest test suite can be configured to enable extra run time validation checks against the skupper router.

Since run time validation slows down skrouterd considerably it is disabled by default.

It can be enabled by setting the RUNTIME_CHECK build flag via the cmake command.

NOTE: Depending on your environment the ctest suite may time out if validation is enabled due to the additional run time overhead it adds. You can extend the default test time via the ctest --timeout option.

.Example [source,shell script]

ctest --timeout 1500 -VV

The Skupper Router test suite supports the following run time validation tools:

==== Valgrind Memcheck

Memcheck runs skrouterd under Valgrind's memcheck leak checker during the CTest suite. This causes tests to fail if a memory error is encountered.
Use the grinder tool (in the bin directory) to create a summary of the errors found during the test run.

The valgrind toolset must be installed in order to use memcheck.

To enable memcheck set the RUNTIME_CHECK build flag to "memcheck":

[source,shell script]

cmake .. -DRUNTIME_CHECK=memcheck

If valgrind detects errors, the skrouterd process exits with an exit code of 42 and a message is displayed in the CTest output. For example:

[source]

RuntimeError: Errors during teardown: Process XXXX error: exit code 42, expected 0

==== GCC/Clang Thread Sanitizer (TSAN) This option turns on extra run time threading verification.

NOTE: Applicable only to GCC versions >= 7.4 and Clang versions >= 6.0.

To enable the thread sanitizer set the RUNTIME_CHECK build flag to tsan:

[source,shell script]

cmake .. -DRUNTIME_CHECK=tsan

The TSAN library (libtsan) must be installed in order to use this option.

If threading violations are detected during the CTest suite the skrouterd process exits with an exit code of 66 and a message is displayed in the CTest output. For example:

[source]

RuntimeError: Errors during teardown: Process XXXX error: exit code 66, expected 0

False positives can be suppressed via the tsan.supp file in the tests directory.

==== GCC/Clang Address Sanitizer (ASAN)

This option turns on extra run time memory verification, including leak checks.

NOTE: Applicable only to GCC versions >= 5.4 and Clang versions >= 6.0.

To enable the address sanitizer set the RUNTIME_CHECK build flag to "asan":

[source,shell script]

cmake .. -DCMAKE_C_FLAGS=-DQD_MEMORY_DEBUG -DRUNTIME_CHECK=asan

On Aarch64, a hardware-assisted address sanitizer is enabled with hwasan.

The ASAN (libasan) and UBSAN (libubsan) libraries must be installed in order to use this option.

[source,shell script]

cmake .. -DCMAKE_C_FLAGS=-DQD_MEMORY_DEBUG -DRUNTIME_CHECK=hwasan

NOTE: The memory pool produces false leak reports unless QD_MEMORY_DEBUG is also defined.

False positive leak errors can be suppressed by using the lsan.supp file in the tests directory.

=== CMake Build Options

Use cmake-gui to explore the CMake build options available. Existing build directory can be opened with cmake-gui -S .. -B .

|=== |CMake option| Description

|-DCMAKE_BUILD_TYPE= |Skupper router defaults to building with the RelWithDebInfo CMake preset. Other options include Debug (disables optimizations) and Coverage.

|-DQD_ENABLE_ASSERTIONS= |Setting this to ON enables asserts irrespective of CMAKE_BUILD_TYPE.

|-DRUNTIME_CHECK= |Enables C/C++ runtime checkers. See "Run Time Validation" chapter above.

|-DSANITIZE_PYTHON=OFF |Enables Leak Sanitizer suppressions for libpython3. Use it to disable sanitization with older Python version (< 3.9).

|-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON |Compiles the project with LTO (Link Time Optimization) enabled.

|-DQD_DISABLE_MEMORY_POOL=ON |Skupper router immediately frees memory, instead of returning it to memory pool. This option breaks safe pointers, resulting in crashes, therefore is suitable only for debugging. When combined with -DRUNTIME_CHECK=asan, the pointer breakages are much less frequent.

|-DBUILD_TESTING=OFF |Excludes project's tests from the build.

|-DVERSION= |Sets the version of skupper-router. E.g. -DVERSION=2.0.0. If not supplied, the version is set to UNKNOWN. The version of skupper-router being used can be obtained running skrouterd --version

|-DBUILD_BENCHMARKS=ON |Benchmarking tests will be built. The libbenchmark library is required by the benchmarks.

|-DENABLE_WARNING_ERROR=OFF |Build will be allowed to succeed when compilation warnings are present.

|===