WiFi connection drops frequently on mesh style network using repeater

[marchingband]

Basic Infos

I have a program using WiFiManager that works very well on most systems, however customers who have a WiFi repeater, mesh network type WiFi system have reported frequent dropped connections.

Hardware

WiFimanager Branch/Release: Master

Esp8266/Esp32:

Hardware: ESP32 WROVER

Core Version: 2.4.0

Description

Problem description

Settings in IDE

Module: Arduino

Additional libraries:

Sketch

#BEGIN
#include "Arduino.h"
#include <stdlib.h>
#include "Audio.h"
#include "WiFi.h"
#include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>
#include "encoder.h"
#include <Fonts/FreeSans18pt7b.h>
#include <Fonts/FreeSans9pt7b.h>
#include <WiFiManager.h> // https://github.com/tzapu/WiFiManager
#include <HTTPClient.h>
#include <ArduinoJson.h>
#include <Preferences.h>

// ENCODER
#define ENC_PUSH 19
#define ENC_A 34
#define ENC_B 35

// OLED
#define OLED_RESET -1 // no reset pin
#define SCREEN_WIDTH 128
#define SCREEN_HEIGHT 32
#define SCREEN_ADDRESS 0x3C
#define DISPLAY_ON_SECONDS 15
Adafruit_SSD1306 display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, OLED_RESET);

// AUDIO
#define I2S_DOUT      23 // 17 // 22  // DIN connection
#define I2S_BCLK      26 // 18 // 26  // Bit clock
#define I2S_LRC       25 // 19 // 25  // Left Right Clock
#define VOLUME_PIN 39
Audio audio;
uint8_t volume = 0;

// STATIONS
#define JSON_HOST "https://marchingband.github.io/campusradioradio/data/stations.json" // canada
// #define JSON_HOST "https://marchingband.github.io/campusradioradio/data/stations-idaho.json" // idaho
Preferences preferences;
struct SpiRamAllocator {
    void* allocate(size_t size) {
        return heap_caps_malloc(size, MALLOC_CAP_SPIRAM);
    }
    void deallocate(void* pointer) {
        heap_caps_free(pointer);
    }
    void* reallocate(void* ptr, size_t new_size) {
        return heap_caps_realloc(ptr, new_size, MALLOC_CAP_SPIRAM);
    }
};
typedef BasicJsonDocument<SpiRamAllocator> SpiRamJsonDocument;
SpiRamJsonDocument *stations;

// VARS
unsigned int current_station;
bool captive_portal_on = false;
bool wifi_connected = false;
bool buffering_audio = false;
bool fetching_stations = false;
unsigned int last_display_update = 0;
bool should_wake_display = false;
int num_stations = 0;

void use_screen(void){
    last_display_update = millis();
}

bool should_sleep(void){
    return(millis() - last_display_update > (1000 * DISPLAY_ON_SECONDS));
}

void readVolume(void)
{
    static int pot = 0;
    static int reads = 0;

    int new_pot = analogRead(VOLUME_PIN);

    // discharge the adc pin
    pinMode(VOLUME_PIN,OUTPUT);
    digitalWrite(VOLUME_PIN,HIGH);
    pinMode(VOLUME_PIN, INPUT);

    // debounce the pot
    if(abs(new_pot - pot) > 186){ // is it relevant?
        if(reads++ < 5){ // not enough in a row
            return;
        } else { // a real reading, continue
            reads = 0;
        }
    } else { // wasnt a relevant read
        reads = 0;
        return;
    }

    uint8_t new_volume = (4096 - new_pot) / 186;

    if(new_volume != volume){
        volume = new_volume;
        pot = 4096 - (new_volume * 186); // place the pot value in the lower bound for that range
        pot += volume >= 21 ? (-98) : 98; // place the pot value in the median value for that range
        audio.setVolume(volume); // 0...21
        use_screen();
        should_wake_display = true;
        log_i("pot: %d %d", pot, volume);
    }
}

void on_radio_encoder(bool up)
{
    use_screen();
    should_wake_display = true;
    int new_station = current_station;
    if(up)
    {
        new_station += (current_station < (num_stations - 1));
    }
    else
    {
        new_station -= (current_station > 0);
    }
    if(new_station != current_station)
    {
        current_station = new_station;
        preferences.putUInt("station", current_station);
    }
}

void configModeCallback (WiFiManager *myWiFiManager)
{
    Serial.println("Entered config mode");
    captive_portal_on = true;
}

static void ui_task(void* arg)
{
    static int last_station  = -1;
    static bool show_dot = false;

    display.begin(SSD1306_SWITCHCAPVCC, SCREEN_ADDRESS);
    display.setFont(&FreeSans18pt7b);
    display.setTextSize(1);
    display.setTextColor(SSD1306_WHITE);
    display.ssd1306_command(SSD1306_SETCONTRAST);
    display.ssd1306_command(0);
    use_screen();
    for(;;)
    {
        if(captive_portal_on)
        {
            use_screen();
            display.setFont(&FreeSans9pt7b);
            display.setTextWrap(false);
            display.clearDisplay();
            display.setCursor(0, 22);
            display.println("SETUP WIFI");
            display.display();
        }
        else if(fetching_stations)
        {
            use_screen();
            display.setFont(&FreeSans9pt7b);
            display.setTextWrap(false);
            display.clearDisplay();
            display.setCursor(0, 18);
            display.println("loading stations");
            display.display();
        }
        else if(!wifi_connected)
        {
            use_screen();
            display.clearDisplay();
            display.display();
        }
        else if(
            (current_station != last_station) ||
            should_wake_display
        )
        {
            should_wake_display = false;
            use_screen();
            last_station = current_station;

            JsonArray data = stations->as<JsonArray>();
            JsonArray station_data = data[last_station].as<JsonArray>();
            const char* station_callsign = station_data[0];

            display.clearDisplay();
            display.setFont(&FreeSans18pt7b);
            display.setCursor(10, 28);
            display.println(station_callsign);
            display.display();
        }
        if( buffering_audio || !wifi_connected || show_dot )
        {
            use_screen();
            display.fillRoundRect(display.width() - 10, display.height() / 2, 6, 6, 3, SSD1306_INVERSE);
            display.display();
            show_dot = !show_dot;
        }
        readVolume();
        if(should_sleep())
        {
            display.clearDisplay();
            display.display();
        }
        vTaskDelay(10);
    }
}

static void audio_task(void* arg)
{
    static int last_station = -1;
    audio.setBufsize(-1, 3000000); // -1 is no ram, 3MB psram (defualt is 300k)
    audio.setPinout(I2S_BCLK, I2S_LRC, I2S_DOUT);
    audio.setVolume(0);
    for(;;)
    {
        // guard against bad station indexes
        if( current_station > num_stations)
        {
            current_station = 0;
        }

        if(current_station != last_station)
        {
            bool connected_to_stream = false;
            while(!connected_to_stream) // keep retrying if connection fails
            {
                last_station = current_station < (num_stations - 1) ? current_station : (num_stations - 1);
                buffering_audio = true;
                JsonArray data = stations->as<JsonArray>();
                JsonArray station_data = data[last_station].as<JsonArray>();
                const char* station_callsign = station_data[0];
                const char* station_host = station_data[1];
                log_i("connecting to %s %s", station_callsign, station_host);
                connected_to_stream = audio.connecttohost(station_host);
                // audio.connecttohost(station_host);
                audio.setVolume(volume);
            }
            buffering_audio = false;
        }
        audio.loop();
    }
}

bool fetch_json(void)
{
    if (WiFi.status() == WL_CONNECTED) {
        HTTPClient http;
        http.begin(JSON_HOST);
        int httpResponceCode = http.GET();
        if (httpResponceCode > 0) {
            Serial.println(httpResponceCode);
            DeserializationError error = deserializeJson(*stations, http.getStream());
            if (error) {
                Serial.print("deserializeJson() failed: ");
                Serial.println(error.c_str());
                http.end();
                return 1;
            }
            num_stations = stations->as<JsonArray>().size();
            log_i("found %d stations", num_stations);
            return 0;
        } else {
            Serial.print("err:");
            Serial.println(httpResponceCode);
            http.end();
            return 1;
        }
        http.end();
        return 0;
    } else {
        Serial.println("wifi err");
        return 1;
    }
}

void get_json(void)
{
    fetching_stations = true;
    stations = new SpiRamJsonDocument(60000);
    while(fetching_stations)
    {
        fetching_stations = fetch_json();
    }
}

void setup()
{
    disableCore0WDT();
    disableCore1WDT();    

    Serial.begin(115200);

    preferences.begin("eeprom", false);
    current_station = preferences.getUInt("station", 0);

    pinMode(VOLUME_PIN, INPUT);
    pinMode(ENC_PUSH, INPUT_PULLUP);

    encoder_init();
    on_encoder = on_radio_encoder;

    xTaskCreatePinnedToCore(ui_task, "ui_task", 4096, NULL, 3, NULL, 1);

    WiFi.mode(WIFI_STA);
    WiFiManager wm;
    // wm.resetSettings();
    wm.setAPCallback(configModeCallback);
    int normal_mode = digitalRead(ENC_PUSH);
    log_i("enc push %s", normal_mode == 1 ? "high" : "low");
    if(normal_mode == 0)
    {
        captive_portal_on = true;
        wm.startConfigPortal("Campus Radio Radio", "campusradio");
        log_i("config closed");
        delay(1000);
        ESP.restart();
    }

    if(!wm.autoConnect("Campus Radio Radio", "campusradio"))
    {
        ESP.restart();
    }
    else
    {
        Serial.println("connected...yeey :)");
        captive_portal_on = false;
        wifi_connected = true;
        get_json();
        readVolume();
        xTaskCreatePinnedToCore(audio_task, "audio_task", 8000, NULL, 3  | portPRIVILEGE_BIT, NULL, 0);
    }
}

void loop(){
    vTaskDelay(1000);
    // vTaskDelete(NULL);
}

void audio_info(const char *info){
    Serial.print("info        "); Serial.println(info);
}
void audio_id3data(const char *info){  //id3 metadata
    Serial.print("id3data     ");Serial.println(info);
}
void audio_eof_mp3(const char *info){  //end of file
    Serial.print("eof_mp3     ");Serial.println(info);
}
void audio_showstation(const char *info){
    Serial.print("station     ");Serial.println(info);
}
void audio_showstreaminfo(const char *info){
    Serial.print("streaminfo  ");Serial.println(info);
}
void audio_showstreamtitle(const char *info){
    Serial.print("streamtitle ");Serial.println(info);
}
void audio_bitrate(const char *info){
    Serial.print("bitrate     ");Serial.println(info);
}
void audio_commercial(const char *info){  //duration in sec
    Serial.print("commercial  ");Serial.println(info);
}
void audio_icyurl(const char *info){  //homepage
    Serial.print("icyurl      ");Serial.println(info);
}
void audio_lasthost(const char *info){  //stream URL played
    Serial.print("lasthost    ");Serial.println(info);
}
void audio_eof_speech(const char *info){
    Serial.print("eof_speech  ");Serial.println(info);
}

#END

Debug Messages

I havn't been able to obtain debug info yet, I just have a few reports, enough to be confident there is an issue, and post here for a solution. Thanks!

[millheat]

We have similar issues were the ESP32 drops its connection and in many cases it wont reconnect until you do a power cycle. This tends to only happen on mesh systems only.

[tablatronix]

How is this related to WM?

[marchingband]

Because flakiness on repeater systems can be improved with changes in the WiFi initialization process which is all in the wm code.

[tablatronix]

What changes?

[marchingband]

It's been a long time, I forget now. They were very technical, and there were lots of slightly different opinions that came up when I did my research online. Are you the library maintainer? I'd be happy to help dig up that research again if it helps.

[tablatronix]

Yes, and I am not familiar with any issue like this, why did you open an issue if you have no details or debug logs?

The only thing I know of that affects wifi, and I have no idea why mesh or repeaters would matter, unless there is some low level phy stuff that can be tweaked, ttl power level, , which can still be done nothing is stopping you.. Is adc and interrupts or known esp bugs like core issues interfering with wifi.

Your code is way to complex to even look into this, but would need a minimal example

[marchingband]

I have no debug logs because like my post says, I havnt reproduced the issue locally. I only have reports from users of my device, and many others online reporting similar issues with repeater networks. I posted the code I have because that's what your issue submission asked for. I don't think anything in my code is the cause. Repeater networks are not very common, and the fix I found was something I thought you would want to implement in your library. If you like, I can do that research again and post some links to the solutions here. Otherwise you can close the issue if you aren't interested.

[marchingband]

https://esp32.com/viewtopic.php?t=17814

[marchingband]

Someone reports that using WiFi.begin(ssid,pass, 0,0,true) fixes the issue. I do not understand why, just FYI.

[tablatronix]

I see, I didn't know what you meant by customers.

Those are the defaults wl_status_t begin(char* ssid, char *passphrase = NULL, int32_t channel = 0, const uint8_t* bssid = NULL, bool connect = true);

[tablatronix]

This sounds like a workaround for an esp bug that affected bssid scanning. hard to tell there were many of them

https://github.com/espressif/arduino-esp32/issues?q=is%3Aissue+WIFI_ALL_CHANNEL_SCAN+is%3Aclosed

[tablatronix]

have you tried

WiFi.setScanMethod(WIFI_ALL_CHANNEL_SCAN); before using wifi

[marchingband]

I havnt tried anything. I may ask one of the users which brand of mesh repeater they are having problems with, purchase it, and go into trying to reproduce and debug. It's a bit beyond my skill set to go into the lower level networking code, so I had hoped you, or someone else here, may know more, or have experience with this situation, and we could perhaps put a fix into the library, so others don't face the issue. These mesh networks are gaining popularity so it could become a bigger issue soon, if Arduino doesn't solve it upstream.

[tablatronix]

I think you just need multiple aps with the same ssid, not necessarily a mesh.

If you sell wifi devices you should defineltly have a wifi lab. I use use cheap linux boxes with wifi adapters and just start up ad hoc networks

[marchingband]

Well that’s a very common scenario, SO many people have some kind of a range extender in the home. If this bug effects all of them that is a big deal indeed!

I have been selling wiFi dev boards for a few years, but only just started selling consumer items, and this is the first time I've encountered an issue. I think you're right I need to improve my WiFi test equipment. In the meantime, are you able to reproduce the issue in your lab? If there is a 1-line fix, would it make sense to patch it into the WM lib?

[tablatronix]

I mean to test this issue, you do not need special mesh hardware I think you only need multiple aps with identical ssids. I can add a option to allow roaming , this is not a fix, but an option in ESP wifi.. but this can just be added to user code or set via a param

[marchingband]

I don't understand enough about the issue to say exactly, but my feeling is that WM should work with repeaters by default, and if there is some other reason the setting should be different (ex. roaming?), it should be set via a config option to WM, with a note in the example code, or somewhere similar, stating that it will break repeater home networks.

It sounds like I could get a few ESP32 modules in AP mode, all advertising with the same network name, and then one ESP32 to join one of those networks, examine logs from this client ESP32, and likely reproduce this bug, is that accurate?

[tablatronix]

I am pretty sure this is just a wifi thing, esp library grabs the first AP and not the best one. Its not a bug its just how it works, to make connections faster. So you have to choose a better scanning option or lock to BSSID

WM has nothing to do with wifi, all esp wifi features are available via the native methods still.

[tablatronix]

I have some ideas though, but ultimately this is an ESP issue.

We really have to let the end user decide which one to choose, we can assume if there are duplicates that this is a mesh and force the option, which is probably the easiest change, the better one is to offer a checkbox in the GUI to lock to that ap BSSID

[tablatronix]

417

[dolanmiu]

FYI,

An issue I opened up is similar to the one in this ticket, I use Google WiFi:

https://github.com/tzapu/WiFiManager/issues/1349

[marchingband]

It seems unlikely to cause anyone problems to implement the easy route you outlined above, a checkbox would be great, but I would think defaulting it to checked (assume it's mesh) would be the best choice. These mesh networks are getting very popular.

[jhbruhn]

Am I right in my understanding that this issue arises (also for me!) from the fact that WiFiManager is storing the credentials via the esp-idf functions and restores them using WiFi.begin(), which stores not only the ssid but also the bssid and thus does not do a rescan on reconnect?

[tablatronix]

There was some changes recently to the default scan method on esp32. So yes wm uses whatever esp arduino does. There is now a parameter to change the scan method for faster connections. But I do not thinnk the bssid is saved unless you specify, I have to recheck this though

I started looking into a reasonable fix for this either way its a pita as we have to pass yet another option though

[gpena208777]

I havnt tried anything. I may ask one of the users which brand of mesh repeater they are having problems with, purchase it, and go into trying to reproduce and debug. It's a bit beyond my skill set to go into the lower level networking code, so I had hoped you, or someone else here, may know more, or have experience with this situation, and we could perhaps put a fix into the library, so others don't face the issue. These mesh networks are gaining popularity so it could become a bigger issue soon, if Arduino doesn't solve it upstream.

Hi, has this issue made any developments? I too am experiencing this issue on Mesh networks. Embarrassing as you say. But in my case the users report they have no connectivity after initial credential saving.

[gpena208777]

Customers router is a Nighthawk Mesh6. The esp32 is visibly connected on the network via the router login page. But a ping returns no packets.