Memory decreasing when using BLEscan

[1technophile]

Hello,

Thanks for all this work on ESP32, a lot of data for working with this really powerfull chip!

I'm working on presence detection or sensor integration with BLE and our ESP32s (BLE to MQTT solution). After a few hours the ESP32 disconnects from the wifi or are stuck (https://github.com/1technophile/OpenMQTTGateway/issues/173)

After some analysis I found what seems to be a memory leak when using BLE scan.

I took the BLE scan example and run it every 20 seconds by checking RAM remaining:

/*
   Based on Neil Kolban example for IDF: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLE%20Tests/SampleScan.cpp
   Ported to Arduino ESP32 by Evandro Copercini
*/

#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEScan.h>
#include <BLEAdvertisedDevice.h>

int scanTime = 10; //In seconds
//Time used to wait for an interval before checking system measures
unsigned long timer_sys_measures = 0;

class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks {
    void onResult(BLEAdvertisedDevice advertisedDevice) {
      Serial.printf("Advertised Device: %s \n", advertisedDevice.toString().c_str());
    }
};

void setup() {
  Serial.begin(115200);
  BLEDevice::init("");
}

void loop() {
  // put your main code here, to run repeatedly:
  delay(2000);

  unsigned long now = millis();
  if (now > (timer_sys_measures + 20000)) {//retriving value of temperature and humidity of the box from DHT every xUL
    timer_sys_measures = millis();
    Serial.println("Remaining memory");
    uint32_t freeMem;
    freeMem = ESP.getFreeHeap();
    Serial.println(freeMem);

    Serial.println("Scanning...");
    BLEScan* pBLEScan = BLEDevice::getScan(); //create new scan
    pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());
    pBLEScan->setActiveScan(true); //active scan uses more power, but get results faster
    BLEScanResults foundDevices = pBLEScan->start(scanTime);
    Serial.print("Devices found: ");
    Serial.println(foundDevices.getCount());
    Serial.println("Scan done!");
  }
}

Here are my results:

Remaining memory
131032
Scanning...
Advertised Device: Name: , Address: xxxxxxxxxx, manufacturer data: xxxxxxxxxx, serviceUUID: xxxxxxxxxx 
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
E (43006) BT: btc_search_callback  BLE observe complete. Num Resp 3

Devices found: 3
Scan done!
Remaining memory
129424
Scanning...
Advertised Device: Name: , Address: xxxxxxxxxx, manufacturer data: xxxxxxxxxx, serviceUUID: xxxxxxxxxx 
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
E (63009) BT: btc_search_callback  BLE observe complete. Num Resp 3

Devices found: 3
Scan done!
Remaining memory
129404
Scanning...
Advertised Device: Name: , Address: xxxxxxxxxx, manufacturer data: xxxxxxxxxx, serviceUUID: xxxxxxxxxx 
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
E (83012) BT: btc_search_callback  BLE observe complete. Num Resp 3

Devices found: 3
Scan done!
Remaining memory
129380
Scanning...
Advertised Device: Name: , Address: xxxxxxxxxx, manufacturer data: xxxxxxxxxx, serviceUUID: xxxxxxxxxx 
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
Advertised Device: Name: Flower care, Address: xxxxxxxxxx, serviceUUID: xxxxxxxxxx
E (103015) BT: btc_search_callback  BLE observe complete. Num Resp 3

Devices found: 3
Scan done!
Remaining memory
129360

We can see that remaining memory decreases over the time.

Any idea about an incorrect implementation in my side? Or maybe you need more infos about this?

[nkolban]

In your source code, you have the following line:

pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());

When ever you see a new call, this allocates storage (in this case for a new instance of a class). When ever you explicitly new data, someone, somewhere has to release it. Have a look at the following code and see if this corrects the issue.

/*
   Based on Neil Kolban example for IDF: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLE%20Tests/SampleScan.cpp
   Ported to Arduino ESP32 by Evandro Copercini
*/

#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEScan.h>
#include <BLEAdvertisedDevice.h>

int scanTime = 10; //In seconds
//Time used to wait for an interval before checking system measures
unsigned long timer_sys_measures = 0;

class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks {
    void onResult(BLEAdvertisedDevice advertisedDevice) {
      Serial.printf("Advertised Device: %s \n", advertisedDevice.toString().c_str());
    }
};

void setup() {
  Serial.begin(115200);
  BLEDevice::init("");
}

void loop() {
  // put your main code here, to run repeatedly:
  delay(2000);

  unsigned long now = millis();
  if (now > (timer_sys_measures + 20000)) {//retriving value of temperature and humidity of the box from DHT every xUL
    timer_sys_measures = millis();
    Serial.println("Remaining memory");
    uint32_t freeMem;
    freeMem = ESP.getFreeHeap();
    Serial.println(freeMem);

    Serial.println("Scanning...");
    BLEScan* pBLEScan = BLEDevice::getScan(); //create new scan
    MyAdvertisedDeviceCallbacks myCallbacks;
    pBLEScan->setAdvertisedDeviceCallbacks(&myCallbacks);
    pBLEScan->setActiveScan(true); //active scan uses more power, but get results faster
    BLEScanResults foundDevices = pBLEScan->start(scanTime);
    Serial.print("Devices found: ");
    Serial.println(foundDevices.getCount());
    Serial.println("Scan done!");
  }
}

[1technophile]

This is really better thanks a lot!!

Remaining memory
129456
Scanning...
Advertised Device: Name: , Address: ***************, manufacturer data: ***************, serviceUUID: ***************
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
E (303052) BT: btc_search_callback  BLE observe complete. Num Resp 3

Devices found: 3
Scan done!
Remaining memory
129456
Scanning...
Advertised Device: Name: , Address: ***************, manufacturer data: ***************, serviceUUID: ***************
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
E (323055) BT: btc_search_callback  BLE observe complete. Num Resp 3

Devices found: 3
Scan done!
Remaining memory
129456
Scanning...
Advertised Device: Name: , Address: ***************, manufacturer data: ***************, serviceUUID: ***************
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
E (343058) BT: btc_search_callback  BLE observe complete. Num Resp 3

Devices found: 3
Scan done!
Remaining memory
129456
Scanning...
Advertised Device: Name: , Address: ***************, manufacturer data: ***************, serviceUUID: ***************
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
Advertised Device: Name: Flower care, Address: ***************, serviceUUID: *************** 
E (363061) BT: btc_search_callback  BLE observe complete. Num Resp 3

[patlunb]

@nkolban Seems to me I have the same issue.

Using Arduino 1.8.5. BLE libraries latest.

Attaching simplified program. MACs and service UUIDs have been hidden

#include "BLEDevice.h"
#define BATTERY_CHAR_UUID 1
#define UNLOCK_CHAR_UUID 2
#define DATA_CHAR_UUID 3

// The characteristic of the remote service we are interested in.
static BLEUUID    serviceUUID("00001204-0000-1000-8000-FFFFFFFFFFFFFFF");
static BLEUUID    batteryCharUUID("00001a02-0000-1000-8000-FFFFFFFFFFFFFFF");  //BATTERY_CHAR_UUID
static BLEUUID    unlockCharUUID("00001a00-0000-1000-8000-FFFFFFFFFFFFFFF"); //UNLOCK_CHAR_UUID
static BLEUUID    dataCharUUID("00001a01-0000-1000-8000-FFFFFFFFFFFFFFF"); //DATA_CHAR_UUID

static BLEAddress *pServerAddress;
static BLERemoteService* pRemoteService;
static BLERemoteCharacteristic* pRemoteCharacteristic;

BLEClient*  pClient;

bool found = false;

std::string operateBLECharacteristicsUUID(byte uuid_to_read)
{
  std::string q;
  if (!pClient->isConnected())
  {
    Serial.println("isConnected == false");
    return q;
  }

  if (uuid_to_read == BATTERY_CHAR_UUID) pRemoteCharacteristic = pRemoteService->getCharacteristic(batteryCharUUID);
  if (uuid_to_read == UNLOCK_CHAR_UUID) pRemoteCharacteristic = pRemoteService->getCharacteristic(unlockCharUUID);
  if (uuid_to_read == DATA_CHAR_UUID) pRemoteCharacteristic = pRemoteService->getCharacteristic(dataCharUUID);

  if ((pRemoteCharacteristic == nullptr) or (pRemoteCharacteristic == NULL))  {
    Serial.print("Failed to find our characteristic UUID");

    return q;
  }

  if (uuid_to_read == UNLOCK_CHAR_UUID)
  {
      uint8_t udata[] = {0xFF, 0xFF};
       if (!pClient->isConnected())
  {
    Serial.println("isConnected == false");
    return q;
  }
       pRemoteCharacteristic->writeValue(udata, sizeof(udata), true);
      return q;
  }

  else
  {
  std::string value = pRemoteCharacteristic->readValue();
  return value;
  }

}

bool connectToServer(BLEAddress pAddress) {
    Serial.print("Forming a connection to ");
    Serial.println(pAddress.toString().c_str());
    pClient= BLEDevice::createClient();
    Serial.println(" - Created client");

    // Connect to the remove BLE Server.
    pClient->connect(pAddress);
    Serial.println(" - Connected to server");

  pRemoteService = pClient->getService(serviceUUID);
  if (pRemoteService == nullptr) {
    Serial.print("Failed to find our service UUID: ");
    Serial.println(serviceUUID.toString().c_str());
    return NULL;
  }
    Serial.println(" - Found our service");
    operateBLECharacteristicsUUID(BATTERY_CHAR_UUID);
}

class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks {
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    Serial.print("BLE Advertised Device found: ");
    Serial.println(advertisedDevice.toString().c_str());
     if ((advertisedDevice.haveServiceUUID()) && (String(advertisedDevice.getServiceUUID().toString().c_str()) == "0000FFFF-0000-1000-8000-FFFFFFFFFFFFFF")) {
      //
     Serial.println("Found our device!");
     found = true;

    } // Found our server
  } // onResult

}; 

void scanBLE()
{
  found = false;   
  BLEScan* pBLEScan = BLEDevice::getScan();
  MyAdvertisedDeviceCallbacks mycb;
  pBLEScan->setAdvertisedDeviceCallbacks(&mycb);
  pBLEScan->setActiveScan(true);
  pBLEScan->start(5);
  pBLEScan->stop();
  if (!found) Serial.println("No sensors found, repeating scan...");

}

void setup() {

 Serial.begin(38400);
  BLEDevice::init("");
  BLEDevice::setPower(ESP_PWR_LVL_P7);
   Serial.println("Starting ESP32 BLE...");
   found = false;
} 

void loop()
{
 while (!found) scanBLE();    

   pServerAddress = new BLEAddress("XX:XX:XX:XX:XX:XX");
   Serial.println("Connecting to sensor");

   if (connectToServer(*pServerAddress)) Serial.println("Connected to the BLE Sensor."); else Serial.println("Connection to BLE Sensor failed");

          if (pClient->isConnected())
          {
            operateBLECharacteristicsUUID(UNLOCK_CHAR_UUID);
            std::string bat_fw_data = operateBLECharacteristicsUUID(BATTERY_CHAR_UUID);
            Serial.println("--------------------------------");
            byte bat = bat_fw_data[0];
            Serial.print("Battery: ");
            Serial.print(bat, DEC);
            Serial.println("%");
             pClient->disconnect();   
             delay(500);
          }

       delete pServerAddress;

   Serial.print("FREE HEAP SIZE: ");
   Serial.println(ESP.getFreeHeap());
   scanBLE();
} 

Here's the getFreeHeap output (after each connection and read cycle):

• FREE HEAP SIZE: 101676
• FREE HEAP SIZE: 95804
• FREE HEAP SIZE: 89924
• FREE HEAP SIZE: 84060

And it goes all the way down to <20000, and then crashes and reboots. Scans without connection (no devices found) doesn't consume any memory, but if the connection is made, then up to 6KB is gone in one go.

Any advice is appreciated, thanks!

[patlunb]

Thanks, the issue was resolved.

Please disregard previous post

[RoyOltmans]

Patlunb I would appreciate the solution...