Suggestions for adding TIPC protocol to net

[lordvictory]

Like what you folks are doing with your coroutine framework and am currently doing a C++20 embedded project (g++ 13.2) that uses TIPC (tipo.io), and would perhaps like to add it to your framework for some investigative work I'm currently persuing. I think it would be a good fit and thought you might have a few tips what to add or modify, as well as thoughts on the matter before I got started.

TIPC is available natively on most Linux distros via a kernel module, and available on several other OS's as well.

TIPC is a transport protocol with some unique features for high speed clustering applications. It typically uses ethernet packets directly (no IP stack needed), but may also be configured to use UDP. Under Linux it uses sockets, but all socket calls are wrapped by a thin system supplied C TIPC library, no io_uring support yet). Addressing is also completely different, and it too is also defined in the TIPC library include file.

TIPC uses reliable datagrams to send and receive message between peers. But the stack has higher level constructs whereby it has a notion of a "service" bus, which connnotes server use and their client users for transactional type processing.

Check out the link above for further information on TIPC.

[jbaldwin]

Hi @lordvictory, thanks for the inquiry, is this the documentation for TIPC? https://docs.kernel.org/networking/tipc.html Currently on the GitHub mobile app and wasn't able to see the link you mentioned.

This sounds like an interesting addition, I would probably suggest by adding a new client under net::tipc namespace to start and see what other net objects can be reused or if we need to augment them, e.g. net::socket. It's not clear yet to me if you need a server or if every client would be more like the libcoro udp client? I've never used this protocol so my best help for now is probably just project organization.

Do you have a possible example or pseudo code on how you would expect a sample program to use tipc via libcoro, that might go a long way in helping determine how it should be implemented.

[lordvictory]

Hi @lordvictory, thanks for the inquiry, is this the documentation for TIPC? https://docs.kernel.org/networking/tipc.html

These docs are the Linux implementation notes. The opther link has better programming information. TIPC is an IETF RFC/Standard and is also implemented on othere systems, but has the best support under Linux. It was originally designed by Ericsson and is used within their own Cellular networks. I came across it a few years ago when I was working in the datacenter for a company that had contracts with Verizon for Enhanced Caller-ID.

TIPC has some unique clustering features I've not seen in other protocol stacks. It is also extremely fast! It may be configured and used in a number of different scenarios, but the one I'm most interested in at the moment is it's Group Communications features. Peers can create a Group whereby unicast messages may be exchanged between peers, or groupcast messages may be broadcast to all peers simultaneously. A groupcast message is like a multicast message, except it is only delivered to group members. The same socket can both send/receive unicast messages, or send groupcast messages.

TIPC also has a Topology Server built into the stack that sends OOB messages when members join/leave the group.

To this end, since my intention (at the moment) is not to design an all encompassiong TIPC interface, but to simply add the interface functionality needed to exercise libcoro using TIPC Group Communications. Once it appears to be stable then perhaps other TIPC features may be introduced.

So, I've opted to produce a libcoro UDP datagram-like interface for TIPC. FIrst thing I noticed what that the definitions for coro::net::domain_t needed to be moved out from that file into a seperate file for a more non-IP protocol approach. I created a file called net.hpp and moved said definition, with newly added TIPC address family. Also moved the to_string() helper there too.

I then proceeded to duplicate the UDP logic, but in tipc directories for the peer include and code files. I also added a new peer method to create a group socket (make_group_socket vs make_socket). Interestingly enough, TIPC does not need to bind() to a socket (like UDP) to get notified via poll() that I/O is ready.

Fortunately, your design for socket() was able to pass the args all the way thru to create the proper TIPC socket type, so I didn't have to call the TIPC library to create the socket...your logic was able to create it correctly. But am calling the TIPC library in several other places internally.

So, I've got the logic compiling and have a small test program and starting to run/debug the changes. WIll let you know how it goes.

[jbaldwin]

Awesome, sorry I've been quite busy the last few weeks and my personal laptop for programming (5 years old!) took a nose dive so I just got a new one. Looking forward to seeing the PR when it is available. It sounds like your design around net/domain/socket are so far fitting in nicely based on your description.

[lordvictory]

Yes, I've got it up and running in a test program. Seems to be operating correctly. I was able to get a Group socket created and send/receive Groupcast or Unicast messages. For my immediate purposes this is what I was after. But this is just a small part of what TIPC is actually capable of.

[lordvictory]

Not sure how to integrate my changes into your repo, I don't often contribute to public source code...last time was probably over 7-8 years ago...an enhancement/fix to gnu libmicrohttpd...and it didn't use github. I simply provided them a copy of the gzip'd changes to the repo, and they folded it in and released it.

In hindsight, I didn't actually need to change much code, or add much new code. Most of my time was spend becoming familair with how you organized the library, as well as trying to figure out how it was put together so I could change/build it simultaneously as I was also writing some test code.

My current work position is as Senior Software Engineer doing custom Embedded Linux ARM development for RFID Readers. In my out-of-tree cross compilation Linux dev environment I created a directory & cmake wrapper above your libcoro-0.12.1 subdirectory, and invoked your cmake file (with no changes to your build system) that would build & install libcoro to a custom location using some command line definitions, like so:

cmake -DLIBCORO_FEATURE_NETWORKING=ON -DLIBCORO_FEATURE_TLS=OFF -DLIBCORO_BUILD_TESTS=OFF -DLIBCORO_BUILD_EXAMPLES=OFF -DLIBCORO_FEATURE_NETWORKING_TIPC=ON -DCMAKE_INSTALL_INCLUDEDIR=inc ..

I added a definition of LIBCORO_FEATURE_NETWORKING_TIPC to conditionally include the TIPC logic. Last time I checked my build built fine also without this definition.

I've only got one "loose end", namely, the TIPC library single include file (tipcc.h), which I lazily just copied over into the include/coro/net directory during development. This file would be a dependency for a TIPC build. It also includes other Linux system include files that further define the TIPC implementation on Linux.

To build libcoro this include file is the only dependency, but to actually build a libcoro using TIPC suuport also requires linking with the TIPC Utils library (https://github.com/TIPC/tipcutils).

I've put together some test code, and it seems to be working fine, but nothing I'd present as an example. But feel it would likely be simple to change your existing UDP test logic to accomodate TIPC, as the TIPC support I added is for reliable datagrams with TIPC Group support. The only difference would be the call to create the group socket. Group datagrams are sent to the Group Address by any member, to be received by all other members.

I could also put together a small standalone TIPC/libcoro example.

Unfortunately, I've no experience with the test system you have set up for this project, nor do I really want to learn it. It's not something I'd likely use again as my day-to-day programming duties don't use those tools.

To use TIPC on Linux one first must make sure that the TIPC protocol stack is loaded and operational. It comes as a Linux Kernel Module, and may be loaded using:

sudo modprobe tipc

The TIPC stack may operate in two different modes: Local or Networked. By default after the above command the TIPC stack is in Local mode (only TIPC processes on the local system may communicate together). To have other systems participate in the TIPC network they too must have their TIPC protocol stack loaded, but and additional command must be used to "bind" the Ethernet Adapter to the TIPC protocl stack. For this, there is a command line tool called tipc (part of the iproute2 package). A typical bind statement might be:

tipc bearer enable media eth dev eth0

where eth0 is the ethernet device name to bind to (use ifconfig to discover if unknown).

Once this is done any TIPC test code may then execute, once build and linked to the TIPC User library, which also may require building if not available in your Linux distro. In my test environment I've already got this built.

Let me know how you'd like to proceed. Perhaps it might be better to communicate this privately?

[jbaldwin]

Unfortunately, I've no experience with the test system you have set up for this project, nor do I really want to learn it. It's not something I'd likely use again as my day-to-day programming duties don't use those tools.

I'm sorry but I won't be willing to merge anything into this repository that is untested. Obviously based on the license you may have your own fork and support it yourself, but from my perspective this is a feature I am unfamiliar with and will not be able to support it moving forward if it does not have tests as I won't know if its broken by new upcoming changes.

I think catch2 is one of the easier C++ test frameworks available and is not an unreasonable requirement for additions to this repository.