Mesh Network implementation

[BlueShrub]

Greetings all.

I'm attempting to use esp-open-rtos to implement a mesh network using the esp8266. At minimum, I'm running into a networking issue when using STATIONAP_MODE, and the AP side and STATION side both are on the same subnet. In this situation, I'm unable to connect to the STATION side after binding to IPADDR_ANY. However, if the STATION and AP interfaces are on separate subnets, this works swimmingly.

The issue is that, when connected to each other, the AP side is going to be on the STATION side of another ESP, so they will be on the same subnet.

I know that Espressif has implemented a mesh network using a specific ROM (ESP-MESH) and they have an SDK for it, but for many reasons (likely the same as yours) I'm trying to steer clear of their software as much as possible.

My question is, am I missing something obvious here? Is it possible to implement a mesh network with esp-open-rtos and the esp-open-sdk?

[BlueShrub]

One possible workaround (and likely not a good solution for various reason) is to define the subnet for each ESP based on the flash_id.

Ex:

struct ip_info ap_ip;
IP4_ADDR(&ap_ip.ip, 10, 10, (id & 0xFF), 1);
IP4_ADDR(&ap_ip.gw, 0, 0, 0, 0);
IP4_ADDR(&ap_ip.netmask, 255, 255, 255, 0);
sdk_wifi_set_ip_info(1, &ap_ip);

ip_addr_t first_client_ip;
IP4_ADDR(&first_client_ip, 10, 10, (id & 0xFF), 2);

dhcpserver_start(&first_client_ip, 4);

[projectgus]

Interesting comment, thanks for getting in touch.

I think you'll always encounter problems trying to place both AP & STATION on the same subnet. They appear as separate network interfaces to LWIP, and it's routing table will always prefer one interface over the other to send out packets - so for this reason you won't be able to route anything to the second network.

Implementing a mesh via AP/STATION modes is an interesting idea. It's probably the only viable option with esp-open-rtos, as we don't have enough knowledge about the lower WiFi layers to craft custom WiFi frames - which is what you probably really want in order to get a normal "mesh" behaviour. AFAIK the only easy path would be to use Espressif's SDK and their mesh implementation, sorry.

Otherwise you'll need to roll your sleeves up and try to further the reverse engineering of the lower MAC and PHY layers. There's already a promiscuous receive mode, but you'll need to figure out how to send arbitrary WiFi frames. That would probably give you enough to build some kind of mesh on top, although it would be a lot of work.

[BlueShrub]

we don't have enough knowledge about the lower WiFi layers to craft custom WiFi frames

Thanks for confirming that -- the more I've learned about how most mesh network implementations operate, the more I realized that lower level control over the hardware is almost always required.

Given my needs, I'm still going to try to figure out a poor-man's-mesh using STATIONAP_MODE. Having tinkered with the Espressif SDKs, I'd still like to steer clear. They could learn a lot from your (and others') work. Putting aside the language barrier involved in trying to understand their API, it's just not written well and it is incredibly cumbersome to use.

Otherwise you'll need to roll your sleeves up and try to further the reverse engineering of the lower MAC and PHY layers.

I would love to try to contribute in this way -- reverse engineering is my jam. The two biggest barriers to this happening are my lack of knowledge of the available tools for this type of platform and (like most of us) time. In this specific instance I'm spreading myself thin trying to develop a proof of concept.

I'll start looking into what all is required to start really diving under the hood to see what, if anything, I can contribute to lower level hardware control as soon as I'm able. In the meantime, if you have a direction you could point me in in terms of useful software for this purpose, I'd be appreciative. I would very much like to add some value to this project and this sounds like a great way to do that.

Thanks for the response :)

[projectgus]

I'll start looking into what all is required to start really diving under the hood to see what, if anything, I can contribute to lower level hardware control as soon as I'm able. In the meantime, if you have a direction you could point me in in terms of useful software for this purpose, I'd be appreciative. I would very much like to add some value to this project and this sounds like a great way to do that.

There's a reverse engineering wiki here: http://esp8266-re.foogod.com/wiki/ESP8266_Reverse_Engineering_Wiki

There's an "advanced objdump" tool here that's my disassembling tool of choice: https://bitbucket.org/foogod/xtobjdis

(Both the above resources are from @foogod, who is also a key esp-open-rtos contributor.)

There are also these tools: https://github.com/pfalcon/ScratchABit/ https://github.com/pfalcon/ScratchABlock

... and radare2 - https://github.com/radare/radare2/ - recently added support for xtensa lx106, so it should be able to disassembly Espressif libraries as well (I've not tried it yet.)

[hrj]

Does esp-open-rtos support ipv6? I have heard that recent versions of lwip have ipv6 support. If so, creating unique subnets for each interface would be pretty easy (mac address is 6 bytes, ipv6 addresses are 16 bytes).

[denkop]

At first, I want to say big thanks you guys. You are made a really amazing and useful ESP framework! ;)

But I have some trouble with STATIONAP_MODE mode. When I turning on STATIONAP_MODE, domain resolving doesn't work. In STATION_MODE everything ok. I cheked this in mqtt_client example: https://github.com/SuperHouse/esp-open-rtos/tree/master/examples/mqtt_client In line 175 change STATION_MODE to STATIONAP_MODE and configure AP mode in line 177 with following code:

    struct ip_info ap_ip;
    IP4_ADDR(&ap_ip.ip, 172, 16, 0, 1);
    IP4_ADDR(&ap_ip.gw, 0, 0, 0, 0);
    IP4_ADDR(&ap_ip.netmask, 255, 255, 0, 0);
    sdk_wifi_set_ip_info(1, &ap_ip);

    struct sdk_softap_config apconfig = {
        .ssid = AP_SSID,
        .ssid_hidden = 0,
        .channel = 3,
        .ssid_len = strlen(AP_SSID),
        .authmode = AUTH_WPA_WPA2_PSK,
        .password = AP_PASSWORD,
        .max_connection = 3,
        .beacon_interval = 100,
    };
    sdk_wifi_softap_set_config(&apconfig);

    ip_addr_t first_client_ip;
    IP4_ADDR(&first_client_ip, 172, 16, 0, 2);
    dhcpserver_start(&first_client_ip, 4);

mqtt client doesb't work. mqtt_network_connect return error -1. This error return follow code:

int  mqtt_network_connect(mqtt_network_t* n, const char* host, int port)
{
    struct sockaddr_in addr;
    int ret;

    if (host2addr(host, &(addr.sin_addr)) != 0)
    {
        return -1;
    }

I try to check this issue in UDK RTOS in similar exmple (https://github.com/baoshi/ESP-RTOS-Paho) and it no have this issue. But code have one difference. In UDK I don't need manual setup ip configuration and dhcp options. I think in UDK some different in implementation AP mode initialization.

Please help to solve this issue. Thanks for the response :)

P.S. Sorry for my bad English!

[devyte]

Going back on topic, I followed the steps outlined by Andreas Reischle (areresearch), and by enabling IP_FORWARDING in lwip, I was able to get the sta and ap interfaces to talk to each other. To test capabilities, I connected an esp to my router, and my laptop to the esp ap, and I was even able to stream netflix for half an hour through the esp without issues. By itself, this solution is not enough to allow meshing, but I think it proves that the esp is fully capable of a full blown mesh implementation at network level.

[denkop]

My issue solved. Maybe someone will be useful. After hours debugging with enabled IP_DEBUG, TCP_INPUT_DEBUG, TCP_OUTPUT_DEBUG and TCPIP_DEBUG I see that lwip try to send packets via en1 iface. en1 this is default softap iface. But internet access available only on en0(default station mode iface), that connected to my access point.

Solution:

1. You can directly call lwip function for setup default iface:

netif_set_default(netif_find("en0"));

It's not best way, but you can don't touch SDK sources.

2. Modify SDK sources. In app_main.c, replace this code:

   if (sdk_g_ic.s.wifi_mode == 3) {
       sdk_wifi_station_start();
       sdk_wifi_softap_start();
       netif_set_default(sdk_g_ic.v.softap_netif_info->netif);
   }

   to this:

   if (sdk_g_ic.s.wifi_mode == 3) {
       sdk_wifi_station_start();
       sdk_wifi_softap_start();
   #ifdef DEFAULT_IFACE_STATION
       netif_set_default(sdk_g_ic.v.station_netif_info->netif);
   #else
       netif_set_default(sdk_g_ic.v.softap_netif_info->netif);
   #endif
   }

   add at top of app_main.c: #include "user_config.h" and add follow line to user_config.h: #define DEFAULT_IFACE_STATION

I think it's a some bug. Because it's a not a typical network configuration, when the client of access point sharing internet. Maybe developers fix it in future.

[miladshoaei]

Hello friends Excuse me that my English language is bad. I want to implement Mesh Networks with esp8266, what should I study to get started? Thanks