Where is DeduceImpl from misc defined?

[lattice0]

DeduceImpl is declared here:

https://github.com/ambrop72/aipstack/blob/master/src/aipstack/misc/Function.h#L260

but never defined in the entire project.

Is this project incomplete? I've haven't had time to look the entire thing, but when reading through I've found that DeduceImpl has no definition.

I'm also working on some commits that add the possibility to use a Virtual Tap (so you can transmit ethernet packets through OpenVPN in TAP mode, which works like a virtual tap), and also add an example client that talks just at the IP level, so IP packets can be transmitted through OpenVPN in TUN mode. My plan is to do a pull request in the near future. And in the more distant future, I plan to add IPv6 support.

So please give me a help.

[ambrop72]

Hi,

DeduceImpl is declared here: but never defined in the entire project.

DeduceImpl is only declared, never defined, since that function is never actually called, it is just used within decltype to deduce the types involved in a pointer to member function. I'm not sure why you are asking about this, are you getting an error, and with which compiler? If you are, a simple fix is to to provide dummy implementations of DeduceImpl (e.g. { return {}; }).

Is this project incomplete?

The project should be in a fully functional state. I've tested the example program (in examples/) on Linux (gcc 7) and Windows (VS2017 I think). I'm using it successfully in an microcontroller project (https://github.com/ambrop72/aprinter).

I'm also working on some commits that add the possibility to use a Virtual Tap and also add an example client that talks just at the IP level

TUN/TAP support already exists for Linux and Windows, the example program makes use of it. See the code in src/aipstack/tap and examples/tap_iface.h. It does indeed only support Ethernet level only. If you want to add IP level support, it's probably best to:

• modify the code in src/aipstack/tap/ to support IP level as well via a flag,
• add a new example program, e.g. aipstack_example_iplevel.cpp, in addition to aipstack_example.cpp, but which reuses example_app.h.

so IP packets can be transmitted through OpenVPN in TUN mode.

I'm not sure what exactly you mean, but you can't use the same TUN/TAP device that OpenVPN manages via the TUN/TAP API. Instead you need to use Linux packet sockets or (win)pcap, or modify OpenVPN to use an IPC socket instead of TUN/TAP, that the aipstack program connects to. Alternatively you can do is create another TUN/TAP device and bridge the two device via OS facilities. On Linux this only works for Ethernet level devices (but I'm guessing something similar can be done for IP-level devices). If you want this approach, you can just use Ethernet level devices.

And in the more distant future, I plan to add IPv6 support.

That would be great.

[ambrop72]

I'm not really sure what you mean by Virtual TAP, is this something else than the TUN/TAP interface that is already used in the example program?

[ambrop72]

BTW, I'm not sure what you're trying to achieve with EventProviderVirtual, but note that to integrate the TCP/IP stack into an application, you don't really need to use EventLoop (or any of the code in src/aipstack/event_loop), and can instead implement the platform interface in src/aipstack/platform/. That's what I do in the microcontroller project I mentioned. See how src/aipstack/platform/HostedPlatformImpl.h is an implementation of the platform interface that uses the EventLoop.

[ambrop72]

And, if you want aipstack to give outgoing packets to your code (to send), and you want your code to give received packets to aipstack (to process), then all you need is create an instance of IpDriverIface (if you want IP-level) or EthIpIface (if you want Eth-level). To achieve this, the TUN/TAP code is completely unnecessary.

[lattice0]

Thank you by your fast and clear response. Now I understand why the code works. I didn't try to compile yet, I'm editing the code slowly and found myself trapped into this part.

I've forgot completely to tell what I'm doing so you can understand the 'virtual tap' thing.

The way OpenVPN3 default client works is that it reads and sends IP/ETH packets directly to a TUN/TAP interface. So it implements TUN/TAP communication for each system separately.

I've created an OpenVPN3 client based on OpenVPN3's code, that instead of using the system's TUN to read/write packets, I simply craft them by hand and send directly. I put packets into a queue, and the OpenVPN3 reads from this queue and sends them through the tunnel, no TAP/TUN required. Since I have to send and read IP packets by hand, because I don't rely on the system (which implements TCP/IP stack), I needed a TCP/IP stack, and I found yours, which is very well written.

Your library has TAP drivers for Linux and Windows, but since I want to plug my OpenVPN3 client to your TCP/IP stack, I don't need those TAP drivers, I need a 'virtual' one, in the sense that the driverSendFrame and frameReceived in my virtual TapIface actually talk to OpenVPN3 instead of talking to the system's TAP. This, of course, is for when OpenVPN3 transports Ethernet packets (OpenVPN3 TAP mode). If the packets being transported are IP (OpenVPN3 TUN mode), then I don't need a 'virtual tap', I can simply send and receive IP packets directly to IpStack. Or I think I can.

I thougth I should simply call IpStack<Arg>::processRecvedIp4Packet with my IP packet. Isn't this enough? No TAP needed.

Now, why did I completely ditch off the system's TCP/IP stack? Wouldn't it be easier for me if I simply send a packet to a TUN/TAP device on the system and let it be read/written by OpenVPN3? Well, on Android (at least), there's no way to create more than one VPN connection at the time (at least I didn't find a way). But I really need to make multiple VPN connections, so I altered OpenVPN3 to work as a library, not as a program that talks to TUN/TAP.

[ambrop72]

I need a 'virtual' one, in the sense that the driverSendFrame and frameReceived in my virtual TapIface actually talk to OpenVPN3 instead of talking to the system's TAP

Indeed this is kind of what I suggested with "IPC socket". But like I wrote in the last comment, ditch the TUN/TAP code, and create an instance of IpDriverIface. With that, you will be able to:

• Give received packets to aipstack to process (call IpDriverIface::recvIp4Packet).
• Get outgoing packets from aipstack to send (via the callback IpIfaceDriverParams::send_ip4_packet).

Regarding the event loop, you basically have two options:

• Use the existing EventLoop. It has all the basic functionalities allowing you do anything you could need, including file descriptor readiness notification (EventLoopFdWatcher), timers (EventLoopTImer) and reacting to events originating from other threads (EventLoopAsyncSignal). With this design, you will need a dedicated thread to run this event loop, and you could only use aipstack APIs (e.g. TCP) from this thread.
• If you want to integrate into an existing event/main loop, then make a custom implementation of the platform API (the event loop code is unnecessary in this case). This mostly involves implementing a clock and timers (and nothing like FdWatcher and AsyncSignal).

[lattice0]

Thank you for your help, I now understand better what you did.

I've modified EchoClient to behave as a client (what you call a client in this example is actually a server for me ). I don't know if I'm supposed to, but I used TcpConnection::startConnection, like this

TcpConnection::startConnection(parent->tcp(),
                                           {
                                            parent->ip4Address,
                                            parent->port,
                                            parent->receiveWindow
                                           });

However it looks like the library was written with server in mind, because a response is sent for each request. So it's actually the eventLoop which sends the request, because it's the eventLoop which triggers dataReceived (which triggers sendPush and dataSent).

The problem is that I can't find a way to make eventLoop trigger a sendPush, and I obviously can't call sendPush from another thread because IpStack is not thread safe. So how should a TCP client (in the sense that it connects to, not accepts connection) work?

[ambrop72]

The library supports client and server equally well. TcpConnection is the API you use to work with a TCP connection, but:

• For client (you are connecting to a server), you initialize TcpConnection by calling startConnection. Then you use that TcpConnection object to communicate with the server.
• For server you create a TcpListener object with a callback that will be called for each incoming connection, and in that callback you initialize a TcpConnection by calling acceptConnection. Then you use that TcpConnection object to communicate with the client.

In any case you need to create a derived class of TcpConnection in order to be able to implement its callbacks which are virtual functions. The TcpConnection class does not know anything about requests and responses, it works on the level of sending/receiving bytes. You give it data to send, it gives you data that was received.

The classes in the example are named Client because an instance represents a connection from a client (except in the case I mention next).

The example app actually does support connecting to a server, it does that if a client connects on port 2002 (LineParsingClient) and sends the command "connect addr:port". In this case an EchoClient is created, but you can easily change the code to make a LineParsingClient. EchoClient/LineParsingClient don't need to be modified to be usable for connect.

But if what you want to do is send a request and wait for a response (the opposite of what LineParsingClient is doing), that is something you will need to code, and the TcpConnection API is not preventing you from doing that.

Yes you can't use any of the library APIs from outside the event loop. If you are not able to make all your code run in the same event loop, than you will need to synchronize between threads. If you are using the AIpStack event loop, and you have another thread generating pieces of data to be sent, you can do this:

• Create a queue of pieces of data to be sent, protected by a mutex, and an instance of EventLoopAsyncSignal.
• When that other thread needs to send something, put the data into the queue (under mutex) then call EventLoopAsyncSignal::signal().
• In the callback you gave to EventLoopAsyncSignal, which will run in the event loop, remove data from the queue (under mutex) and send it via the TcpConnection APIs (write data to buffer, call extendSendBuf, call sendPush). Use a loop to send data until there is no more in the queue (don't sent just one because it is not guaranteed that the callback is called once for each signal() call, only that it's called soon after signal() is called). I'm assuming you are able to put all this data into the TCP send queue, if not then it's a bit more complicated.

If you could explain where you are getting the data to send, and what you're doing with received data, I may be able to help you do it all in the AIpStack event loop, so there would be just one thread.

[lattice0]

Thanks. You mean calling eventLoop.signal() since it inherits EventLoopAsyncSignal? Is it thread-safe? Also, correct if I'm wrong: I can completely ditch the Linux signal watcher, EventLoopAsyncSignal has nothing to do with them, rigth? Because I need to make platform independent code, so I'm gonna ditch SignalWatcher and other platform related things (just to implement a virtual one in the future using shared mutex for signaling).

I also couldn't find a way to pass a callback to EventLoopAsyncSignal (EventLoop in fact) without altering EventLoop to include a callback binder in EventLoop constructor. Do you mean I need to change this in the library?

I'm also lost in the sea of code that is PlatformRef. I couldn't understand it's meaning, could you talk a little about it?

[ambrop72]

You mean calling eventLoop.signal() since it inherits EventLoopAsyncSignal?

EventLoop doesn't inherit EventLoopAsyncSignal and doesn't have a signal(), not sure where you are getting this from.

EventLoopAsyncSignal is a class whose purpose is to allow you to run an arbitrary function in the event loop thread upon request from an arbitrary thread. You construct an instance of EventLoopAsyncSignal (within the event loop thread) with some callback function. Then you can call EventLoopAsyncSignal::signal() on this object from any thread (this function is thread-safe), and each time you do this, the callback will be executed in the event loop (careful though: not necessarily once for each signal() call, e.g. you could call signal() twice in quick succession and the callback may only run once after the second call).

EventLoopAsyncSignal has nothing to do with Linux / C signals, it is a thread synchronization mechanism.

I also couldn't find a way to pass a callback to EventLoopAsyncSignal.

It's the second argument to the constructor.

I'm also lost in the sea of code that is PlatformRef. I couldn't understand it's meaning, could you talk a little about it?

Sure. AIpStack has a platform abstraction layer (src/aipstack/platform) whose purpose is to allow one to integrate the library into almost any application, be it under a regular OS, an RTOS or an OS-less environment like in microcontroller projects. The core assumption here is that you have a single-threaded environment from the perspective of AIpStack. This means that the following all happen in the same thread:

• Callbacks from Platform API facilities (really just Timer callbacks, see below) into AIpStack.
• Public API calls from the application into AIpStack, including the TCP APIs (e.g. TcpConnection) and network interface APIs (e.g. IpDriverIface::recvIp4Packet).
• Public API callbacks from AIpStack to your application (e.g. TcpConnection::dataReceived and send_ip4_packet given to IpDriverIface). This is really consequence of the above two, since AIpStack does not create any threads of its own, calls from AIpStack to the application just couldn't come from any other thread.

One implements the Platform API by writing a custom class that provides all the definitions that are documented for the PlatformImplStub class (this class is not usable code, just a template for what you need to do). Such a class is commonly called the PlatformImpl class (especially when it is a template parameter).

The PlatformImpl class must, most importantly, provide the following:

• Clock (e.g. TimeType time, TimeFreq constant, getTime function).
• The Timer class which allows AIpStack to be notified when the clock has reached a particular absolute value. AIpStack creates and uses timers for various purpose, e.g. for timeouts in ARP, TCP and DHCP.

PlatformRef is a very simple template class, templated by PlatformImpl. Assuming that PlatformImpl defines ImplIsStatic to false, PlatformRef is constructed from a pointer to the PlatformImpl and internally stores this pointer (otherwise, PlatformRef doesn't take or store such a pointer since all the PlatformImpl functions must be static) - see here.

PlatformFacade is an internal wrapper around a PlatformImpl through which AIpStack accesses the PlatformImpl. Its purpose is to verify statically that PlatformImpl provides the correct definitions, including that the types of function arguments and return values are correct. PlatformFacade is constructed from a PlatformRef. Your application only needs to construct a PlatformFacade from PlatformRef and pass it to whatever requires you to provide it, e.g. IpStack().

The AIpStack EventLoop (together with associated classes like EventLoopAsyncSignal and EventLoopFdWatcher) is a generic event loop implementation, similar to libuv, libev, libevent, GLib event loop.... It is not coupled to the TCP/IP code and can be used stand-alone. If you want to use this EventLoop together with the TCP/IP, you use HostedPlatformImpl as your PlatformImpl, you do not write your own PlatformImpl. This is exactly what aipstack_example does. However if you do not want to use the AIpStack EventLoop, you have to provide a custom PlatformImpl.