[6LoWPAN][PoC] Integrate picoTCP with RIOT-OS

[phalox]

RIOT-OS is an interesting OS for the Low Power IoT devices. They currently have a good IPv6 and 6LoWPAN implementation, but lack TCP or IPv4 features which could very well be interesting on devices that have to act as gateways. Having one and the same stack will allow developers to create their apps for all these devices in one go.

Therefore, we would like to have a port of picoTCP in RIOT-OS, replacing their network stack. From what the RIOT developers told me, this should be quite easy, seen their modular design.

http://www.riot-os.org/

[toonst]

To get some inspiration on how to integrate picoTCP with gnrc (the existing RIOT stack), have a look at the lwip integration: https://github.com/RIOT-OS/RIOT/pull/3551

[miri64]

Just some correction to dampen some confusion you might get when porting picoTCP: The PR you are referring to is not integrating lwIP with GNRC. It puts it between the conn API (our lightweight "socket" API) and the netdev2 API (RIOT's network device HAL). GNRC does the same, so this PR allows you to replace lwIP with GNRC if you wish to.

[OlegHahm]

http://summit.riot-os.org might be an interesting occasion for some of you to discuss about these efforts and meet other folks from the RIOT community. :)

[toonst]

@authmillenon thanks for the clarification. @OlegHahm I would love to come. We'll see what we can do. :)

[basilfx]

I talked to @phalox yesterday and I am willing to look into this. It is a nice opportunity to gain knowledge about the internals (as part of the plan to bring picoTCP knowledge to ISY NL).

I have just ordered an Archo Pro to get started. This board is not yet supported by RIOT-OS, but the LPC1768 MCU is. Therefore, it should be easy to add it (I do have some experience with RIOT-OS).

[OlegHahm]

Cool, looking forward to it. Does the Archo Pro does provide any transceiver?

[toonst]

@basilfx Cool! Keep us up to date, I might be able to help if you find any impediments.

[phalox]

@OlegHahm The arch pro boards are in the Arduino form factor and come with an ethernet jack and all magnetics. We are already running picoTCP on these boards, so the combination with Riot would be totally awesome!

[basilfx]

I have received my Arch Pro last week. I'll see if I can find the time this week to add initial support for the board to RIOT-OS.

[basilfx]

Initial support is in my tree, based on the mbed LPC1768 board.

It uses OpenOCD for flashing, as it was already supported by RIOT-OS. The firmware needs a checksum, for which I (currently) use something I wrote before.

[phalox]

Nice! Looking forward to more cool stuff :-)

[basilfx]

Support for Seeeduino Arch Pro was merged in RIOT-OS (https://github.com/RIOT-OS/RIOT/pull/5555). It should now be possible to run BOARD=seeeduino_archo-pro make to compile.

I believe @toonst had some time left in Berlin to start working on the picoTCP netdev2 driver :-)

[basilfx]

Found some time to work on this again. I followed up on the work of @toonst, but took a different route: I wrote a picoTCP device driver that wraps a RIOT-OS netdev2 driver. Don't know if this is the best approach, but I got it working with the netdev2_tap driver running native.

So here is the first (and only) packet being sent from picoTCP within RIOT-OS. Next stop: receiving packets :-P

[phalox]

That's great! Could you tell a bit more about the architecture? I'm not familiar with the RTOS netdev devices!

[basilfx]

Sure. In RIOT-OS, network interfaces (wireless, wired, emulated) can implement a so-called netdev2 driver. This driver exports a some methods and can be configured using get/set methods using pre-defined options.

In RIOT-OS, the default GNRC network stack connects to a netdev2 driver via a wrapper. The picoTCP device driver is similar to this wrapper. So when you call pico_netdev2_create(..), you invoke it with a netdev2 driver, name and MAC address. Then it will configure the MAC address and ISR callback. When picoTCP wants to emit a frame, this frame is passed onto the netdev2 send method. Of course, the underlying device must accept ethernet frames. For the user, you'll just keep using the picoTCP API.

I'll polish the code a bit before I will push it to Github. It is still based on @toonst own fork of picoTCP, so I have to figure out what is actually required from his fork. I'm not sure if this is the best approach, but I think that I just have to see where this will end :-)

One problem I have to figure out has to with memory management: RIOT-OS doesn't ship with malloc/free, so I am using the picoTCP memory manager. But internally, this memory manager is still using malloc/free. RIOT-OS does have a oneway_malloc module that does exactly what it is named after, but instead I would like to give picoTCP a chunk of memory at compile time and forget all about memory allocation. Any idea's?

[frederikvs]

Nice work on the port :-)

Regarding the memory management problems : currently, the memory manager is mostly (only?) used by some of our tests. I'm not aware of any production code using it.

Internally it does still depend on a malloc-style function, but it only calls this function with a fixed size PICO_MEM_PAGE_SIZE. This means the implementation can be far simpler than a complete malloc : a range of memory, and a map of which blocks have been allocated, should be enough. It would be nicer if this functionality was also included in picoTCP itself, so that you can truly just give it a static block of memory, and it'll handle it all. At the time the mm was written, the main goal were the tests however, so work was stopped a little short of a full-blown memory manager.

If you feel like implementing the block-allocation, please feel free to do so ;-). If you don't, I'll probably put it in the backlog somewhere, see if/when somebody has the time for it. Of course, we will need to spend a few moments to consider the "interface" of this... A few externs that the user needs to provide, or a pico_mm_init which gets passed the necessary pointer and size (to be called before anything else at all!), or something else entirely.

A bit more detail on the implementation, though this is from memory, so I don't guarantee correctness. Normally, everywhere in our stack we should be using PICO_ZALLOC and its capital friends, which can be mapped either to the user-provided pico_zalloc (when running without our mm), or to the functionality of pico_mm.c which then calls the user-provided pico_zalloc with fixed block sizes, as mentioned before.

We should make a new issue for this.

P.S. note that it's not at all strange for a memory manager to depend on another, simpler memory allocation system. I think most implementations of malloc depend on stuff like sbrk to provide it with memory to dole out.

[basilfx]

@frederikvs Thanks for the explanation. I already assumed that using pico_mm, the allocations would be constant. So here is my dirty implementation:

#include <string.h>
#include <stdio.h>

#include "bitfield.h"
#include "assert.h"

#define PICO_MEM_PAGE_SIZE 4096

static uint8_t pages[1];
static uint8_t memory[PICO_MEM_PAGE_SIZE * 4];

void *pico_zalloc(size_t size)
{
    (void) size;

    assert(size == PICO_MEM_PAGE_SIZE);

    int index = bf_get_unset(pages, 4);

    if (index == -1) {
        puts("zalloc: failed\n");
        return NULL;
    }

    puts("zalloc: ok\n");
    return memset(&memory[PICO_MEM_PAGE_SIZE * index], 0, PICO_MEM_PAGE_SIZE);
}

void pico_free(void *ptr)
{
    uint8_t index = (uint8_t) (((int) ptr) - ((int) memory)) / 4;

    puts("free: ok\n");
    bf_unset(pages, index);
}

And yes, this works: I got ping, DHCP, and a webserver running!

[frederikvs]

nice work!

Regarding your quick implementation of the page-allocator : great that it works for your application, but unfortunately, it doesn't look like it's all that portable yet. If we want to include it in the stack, it needs some work. I think stuff like the bf_get_unset is RIOT-specific? Let me open a new issue to track this. If you feel like getting that page-allocator up to our normal standards, feel free to assign the issue to yourself, if you don't, I'll see if/when we get somebody else available for it :-)