Wifiprov does not work correctly (scan does not show any wifi network)

[0x0fe]

Board

ESP32D0WR2

Device Description

NA

Hardware Configuration

NA

Version

v2.0.9

IDE Name

Arduino

Operating System

windows 10

Flash frequency

80M

PSRAM enabled

yes

Upload speed

921600

Description

So, using the provided example for wifiprov, and the app ESP BLE Prov app from appstore (downlaoded minutes ago) i can connect to the device via BLE, no problem, but the wifi networks scan doesnt work, no network is displayed at all. If i type the SSID manually from "Join Other Netowrk" then it can connect and the process finishes normally. But impossible to get any network scanned, which is a proble of course because in most case we dont know the exact SSID, or it is inconvenient to type.

Sketch

wifi prov default example, set to BLE mode

Debug Message

no error

Other Steps to Reproduce

Load the wifi prov example, install ESP BLE Prov on a iPhone, reproduce, rinse and repeat.

I have checked existing issues, online documentation and the Troubleshooting Guide

• [X] I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[SuGlider]

@sanketwadekar - I has also tried it and I also can't make it work... Could you please take a look?

[arkhipenko]

Same here. Used to work before. It does not matter what provision scheme handler you use - all three do not work:

WIFI_PROV_SCHEME_HANDLER_FREE_BT
WIFI_PROV_SCHEME_HANDLER_FREE_BTDM
WIFI_PROV_SCHEME_HANDLER_FREE_BLE

[arkhipenko]

UPDATE: when I run it with -D CORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_VERBOSE max logging level - it WORKS! Does not work if the logging is disabled or any lower debug levels for that matter.

Suspect someone did not test the release code... it worked in debug

[0x0fe]

sorry for the lack of updates on this ticket, here it works now (in release mode). the wifi scan works everytime. I will paste the current code below.

[0x0fe]

working test code for prov :

static void WiFiEvent(arduino_event_id_t event){

  //printf("WiFi-Event %d:\n", event);

  switch (event) {

    case ARDUINO_EVENT_PROV_INIT:
      printf("Prov: Init\n");
    break;
    case ARDUINO_EVENT_PROV_DEINIT:
      printf("Prov: Stopped\n");
    break;
    case ARDUINO_EVENT_PROV_START:
      printf("Prov: Started\n");        
      prov_on_flag=1;
    break;
    case ARDUINO_EVENT_PROV_END:
      printf("Prov: End\n");
      prov_end=1;
    break;
    case ARDUINO_EVENT_PROV_CRED_RECV:
      printf("Prov: Credentials received\n");
    break;
    case ARDUINO_EVENT_PROV_CRED_FAIL: //38
      printf("Prov: Credentials failed\n");   
      provError();    
      prov_err=1;
    break;             
    case ARDUINO_EVENT_PROV_CRED_SUCCESS:
      printf("Prov: Credentials success\n");
      provSuccess();
    break;
    case ARDUINO_EVENT_WIFI_SCAN_DONE:
      printf("Prov: scan done\n");
    break;
    default:
    break;
  }
}
esp_err_t custom_prov_data_handler(uint32_t session_id, const uint8_t *inbuf, ssize_t inlen, uint8_t **outbuf, ssize_t *outlen, void *priv_data)
{
  printf("custom_prov_data_handler\n");
    if (inbuf) {
        //ESP_LOGI(TAG, "Received data: %.*s", inlen, (char *)inbuf);
        printf("Received data: %d %s\n", inlen, (char *)inbuf);
    }
    char response[] = "SUCCESS";
    *outbuf = (uint8_t *)strdup(response);
    if (*outbuf == NULL) {
        //ESP_LOGE(TAG, "System out of memory");
        return ESP_ERR_NO_MEM;
    }
    *outlen = strlen(response) + 1; /* +1 for NULL terminating byte */

    return ESP_OK;
}
void init_prov(void){

  configTime(gmtOffset_sec, daylightOffset_sec, ntpServer);
  WiFi.onEvent(WiFiEvent);
  String name = String("faba_");
  name += &MAC_ASCI[7];
  printf("prov name: %s\n",name.c_str());

  WiFiProv.beginProvision(WIFI_PROV_SCHEME_BLE, WIFI_PROV_SCHEME_HANDLER_FREE_BLE, WIFI_PROV_SECURITY_1, "xxxx8888", name.c_str()); 
  wifi_prov_mgr_endpoint_register("custom-data", custom_prov_data_handler, NULL);
  wifi_prov_mgr_disable_auto_stop(1000);
}
void prov_handler(void){

  if(prov_on_flag && poweron_end_flag){
    poweron_end_flag=0;
    prov_on_flag=0;
    provOn();
  }
  if(prov_end){
    if(cnt3++>=33333)
    {
      prov_end=0;
      cnt3=0;
      printf("prov stopped\n");
      wifi_prov_mgr_stop_provisioning();
    }
  }
  if(prov_err){
    if(cnt2++>=10000)
    {
      prov_err=0;
      cnt2=0;         
      WiFi.disconnect(true, true);
      ESP.restart();
    }
  } 
}

[0x0fe]

so the most import points : -In case of error, the best option is to clear credentials (if any) and restart the ESP32, otherwise, in my experience it quickly misbehave when we try to restart the provision from the app.

-a timeout is required in case of error and when provisioning ends, prior to resetting or stopping the provisioning process, otherwise the app will hang forever waiting for some status update (it seems slow to update, because we can see the failure on the LED for example several seconds before the app reacts, on a recent iphone).

-there seems to be some problems with the custom-data mechanism, first, it is not possible to declare it in the application code with wifiProv wrapper, the only way to declare it correctly is inside the wifiProv wrapper itself, i added it line 113

Once declared here we can call wifi_prov_mgr_endpoint_register("custom-data", custom_prov_data_handler, NULL); just after WiFiProv.beginProvision, without error.

The endpoint is apparently registered, there is no error, my colleagues can see it via BLE however they cannot get any data from it .

[arkhipenko]

Hi, @0x0fe I have implemented your suggestion, but it still shows networks ONLY if compiled with -D CORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_VERBOSE Any other level and I get the attached picture...

running on:

 - framework-arduinoespressif32 @ 3.20009.0 (2.0.9)
 - framework-espidf @ 3.40404.0 (4.4.4)

[0x0fe]

@arkhipenko well, i cannot do the debug in your place, it works for us since the changes I mentioned. Anyway, when it works in debug and not in release you already know its a matter of race condition or timing, You can add debug traces in the underlaying IDF api for the wifi provisioning and trace what occurs and what doesnt, there is a set of function for the wifi scan, you can add some printf here and see what is going on.

[mianos]

I have about 10 different ESP modules and I now came across this with the FreeNove Cam that has the S3 module on it. The WifiManger library does not support this board so I went back to the native WifiProv. At the moment I can provision, with wifi, on all the boards except this one. I have reduced my test application to the following, cribbed from another bug and expanded. What I found on my S3 is, if I keep trying to refresh the APs on the app, often it eventually comes up with a list. Sometimes 10 tries. Once I can select one it works. If I enable -D CORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_VERBOSE, as above it works.

100% there is a race condition. I'll try and add some debugging in the wifi scan functions if I can find them, and look into this a bit further.

 - framework-arduinoespressif32 @ 3.20011.230801 (2.0.11) 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-riscv32-esp @ 8.4.0+2021r2-patch5 
 - toolchain-xtensa-esp32s3 @ 8.4.0+2021r2-patch5

[env]
monitor_speed = 115200
platform = espressif32
framework = arduino
board_build.partitions =  min_spiffs.csv

[env:freenove_esp32_s3_wroom]
board = freenove_esp32_s3_wroom
upload_port = /dev/tty.wchusbserial54E20229181
; build_flags =
;    -D CORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_VERBOSE

#include <esp_wifi.h>
#include "WiFiProv.h"
#include "WiFi.h"

void SysProvEvent(arduino_event_t *sys_event)
{
    switch (sys_event->event_id) {
    case ARDUINO_EVENT_WIFI_STA_GOT_IP:
        Serial.print("\nConnected IP address : ");
        Serial.println(IPAddress(sys_event->event_info.got_ip.ip_info.ip.addr));
        break;
    case ARDUINO_EVENT_WIFI_STA_DISCONNECTED:
        Serial.println("\nDisconnected. Connecting to the AP again... ");
        break;
    case ARDUINO_EVENT_PROV_START:
        Serial.println("\nProvisioning started\nGive Credentials of your access point using \" Android app \"");
        break;
    case ARDUINO_EVENT_PROV_CRED_RECV:
        Serial.println("\nReceived Wi-Fi credentials");
        Serial.print("\tSSID : ");
        Serial.println((const char *) sys_event->event_info.prov_cred_recv.ssid);
        Serial.print("\tPassword : ");
        Serial.println((char const *) sys_event->event_info.prov_cred_recv.password);
        break;
    case ARDUINO_EVENT_PROV_CRED_FAIL:
        Serial.println("\nProvisioning failed!\nPlease reset to factory and retry provisioning\n");
        if(sys_event->event_info.prov_fail_reason == WIFI_PROV_STA_AUTH_ERROR)
            Serial.println("\nWi-Fi AP password incorrect");
        else
            Serial.println("\nWi-Fi AP not found....Add API \" nvs_flash_erase() \" before beginProvision()");
        break;
    case ARDUINO_EVENT_PROV_CRED_SUCCESS:
        Serial.println("\nProvisioning Successful");
        break;
    case ARDUINO_EVENT_PROV_END:
        Serial.println("\nProvisioning Ends");
        break;
    case ARDUINO_EVENT_PROV_INIT:
        Serial.printf("Prov: Init\n");
        break;
    case ARDUINO_EVENT_PROV_DEINIT:
        Serial.printf("Prov: Stopped\n");
        break;
    case ARDUINO_EVENT_WIFI_SCAN_DONE:
        printf("Prov: scan done\n");
        break;
    default:
        Serial.printf("Some other event %d\n", sys_event->event_id);
        break;
    }
}

void setup() {
  WiFi.mode(WIFI_STA);
  esp_wifi_restore();
  Serial.begin(115200);
  //Sample uuid that user can pass during provisioning using BLE
  /* uint8_t uuid[16] = {0xb4, 0xdf, 0x5a, 0x1c, 0x3f, 0x6b, 0xf4, 0xbf,
                   0xea, 0x4a, 0x82, 0x03, 0x04, 0x90, 0x1a, 0x02 };*/
  WiFi.onEvent(SysProvEvent);
  WiFiProv.beginProvision(WIFI_PROV_SCHEME_SOFTAP, WIFI_PROV_SCHEME_HANDLER_NONE, WIFI_PROV_SECURITY_1, "abcd1234", "Prov_123");
}

void loop() {
}

[arkhipenko]

Thank you, @mianos for confirming this is not isolated to "my place".

[rbooth84]

I traced out the issue.

WiFiProvClass::beginProvision() --> wifiLowLevelInit() --> tcpipInit() --> _start_network_event_task() Registers the event queue and setups the events with esp_event_handler_instance_register

In _arduino_event_cb() when it sees a WIFI_EVENT_SCAN_DONE it fires postArduinoEvent() with an ARDUINO_EVENT_WIFI_SCAN_DONE event which sends it to the _arduino_event_queue that is picked up by _arduino_event_task() which turns around and sends that to the WiFiGenericClass::_eventCallback().

WiFiGenericClass::_eventCallback() then see's the ARDUINO_EVENT_WIFI_SCAN_DONE event and calls WiFiScanClass::_scanDone() which calls esp_wifi_scan_get_ap_num and esp_wifi_scan_get_ap_records before the provision manager can retrieve them freeing the memory.

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/network/esp_wifi.html#_CPPv419esp_wifi_scan_startPK18wifi_scan_config_tb

If this API is called, the found APs are stored in WiFi driver dynamic allocated memory and the will be freed in esp_wifi_scan_get_ap_records, so generally, call esp_wifi_scan_get_ap_records to cause the memory to be freed once the scan is done

[arkhipenko]

Thank you, @rbooth84! Great work

So, not very elegant, but this patch solves the issue:

esp_err_t WiFiGenericClass::_eventCallback(arduino_event_t *event)
{
    static bool first_connect = true;

    if(!event) return ESP_OK;                                                       //Null would crash this function

    log_d("Arduino Event: %d - %s", event->event_id, WiFi.eventName(event->event_id));
    if(event->event_id == ARDUINO_EVENT_WIFI_SCAN_DONE) {

vTaskDelay(200);  // <============ INSERT SMALL DELAY TO LET WIFIPROV GRAB NETWORKS

        WiFiScanClass::_scanDone();
    } else if(event->event_id == ARDUINO_EVENT_WIFI_STA_START) {
        WiFiSTAClass::_setStatus(WL_DISCONNECTED);
        setStatusBits(STA_STARTED_BIT);
        if(esp_wifi_set_ps(_sleepEnabled) != ESP_OK){
            log_e("esp_wifi_set_ps failed");
        }

[0x0fe]

@rbooth84 @arkhipenko nice @sanketwadekar this wrapper / library is in need of rework.

[rbooth84]

Here is a cleaner fix

esp_err_t WiFiGenericClass::_eventCallback(arduino_event_t *event)
{
    static bool first_connect = true;
    static bool provisioning = false;

    if(!event) return ESP_OK;                                                       //Null would crash this function

    log_d("Arduino Event: %d - %s", event->event_id, WiFi.eventName(event->event_id));

    if(event->event_id == ARDUINO_EVENT_PROV_START) {
        provisioning = true;
    } else if(event->event_id == ARDUINO_EVENT_PROV_END) {
        provisioning = false;
    } else if(event->event_id == ARDUINO_EVENT_WIFI_SCAN_DONE) {
        if(!provisioning) {
            WiFiScanClass::_scanDone();
        }
    } else if(event->event_id == ARDUINO_EVENT_WIFI_STA_START) {
        ...

[mianos]

Does it matter that WiFiScanClass::_scanDone(); is not ever called? Won't the memory not be deleted? Or is it called again later?

[rbooth84]

The provision manager api is calling it and retrieving the values clearing the memory.

[0x0fe]

@rbooth84 this fixes the wifi scan race condition. there is still the problem that the BLE PROV cuts the connection to the mobile before sending wifi connection status, so there is another race condition during this process, it should wait that the mobile has received the wifi connection status before stopping BLE PROV. Also i noticed that when BLE PROV cuts early (before sending confirmation) the app hangs forever, there should be a timeout on the app side.

[mianos]

Any news on this? I keep patching the esp arduino layer with the code from above. I just looked at the head and it does not have this and the scan fails for me on a traditional ttgo esp devkit board.