BLE_client sample crashes with own service UUID

[ghost]

Hi, thank you for providing the BTLE C++ Api. To work with the nativ ESP-IDF it's like try to build a skyscraper but playing in the sand.

I have checked your BLE_client sample in the Arduino IDE. It scans well. If I replace the service UUID with a own one (found before) it crashes at scan->stop. This happens in line

84: advertisedDevice.getScan()->stop();

and also after the 30 seconds scan timeout if I comment out the line 84.

It doesn't find the UUID but crashes.

I have tried it with:

static BLEUUID serviceUUID("0000fe95-0000-1000-8000-00805f9b34fb"); //Flower care
static BLEUUID serviceUUID("0000aa80-0000-1000-8000-00805f9b34fb"); //Texas instruments CC2650 SensorTag

Peter

Editor: Moved to Pastebin: https://pastebin.com/G9HY4cyT

[nkolban]

Howdy my friend, Time for us to roll up our sleeves and find out what is going on. The first thing ... in Arduino land, we have a utility that decodes the back trace to tell us the source code that threw the exception ... see:

https://github.com/me-no-dev/EspExceptionDecoder

Get that working and we'll find out where in the code the problem is occurring.

Next, we can switch on BLE C++ trace ... see:

https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/ArduinoBLE.md

and the section called Switching on debugging.

[chegewara]

It can be problem with this tag (just guessing) https://github.com/sandeepmistry/node-sensortag/issues/45#issuecomment-118979119

[nkolban]

Did you post to the wrong issue?

[chegewara]

No, its same issue with same CC2650 SensorTag, just in different framework. I think it can be connected and related with tag not sending advertise data. As i understand they keep trying to scan with SensorTag.SCAN_DUPLICATES = true; or maybe im just a bit tired

[ghost]

Ok, some more info. The CC2650 SensorTag has the newest FW, also it's only one of more test objects. I use the Arduino IDE 1.8.4 on Win7 64bit with ESP32_BLE.zip Sep 10, 2017.

In the #define sdkconfig.h there is no CONFIG_LOG_DEFAULT, but a CONFIG_LOG_DEFAULT_LEVEL ... changed to 5. Good to know the the ExceptionDecoder also works with ESP32 ;)

Hope it helps ! :-) Peter

Editor: Moved to Pastebin: https://pastebin.com/RqWSkYKW

[nkolban]

Working this now ...

First change made has been the addition of logging the "type" of undefined type that is being logged in the advertising responses as an "unhandled type". Once that fix is pushed, we will then be able to at least learn more about the type that was unhandled beyond "one wasnt handled".

[nkolban]

I had a look at the client source code and am puzzled as the path shown in the trace shouldn't be possible in the sample I provided. Can you pastebin the exact source you are using that manifests this problem?

[ghost]

It's exact the source you provided. For the test I get it new from examples. Only the line with the UUID was changed... see below.

Tell me if I can do further tests, Peter

// The remote service we wish to connect to.
//static BLEUUID serviceUUID("91bad492-b950-4226-aa2b-4ede9fa42f59"); //enabled in 1. run
static BLEUUID serviceUUID("0000aa80-0000-1000-8000-00805f9b34fb"); //enabled in 2. run

[ghost]

what's surprising for me the whole time is, that we can get a service UUID before we have a connection (line 77) !? Update: OK - there is one (GAP), but this one isn't available in connected mode (GATT). And we can not postulate that.

The normal case I know is to: scan, find a match by name or mac, (stop scanning,) connect, read GATT (services, characteristics, values, descriptions), (set notifications,) ...

OK, I have changed the match from serviceUUID to deviceUUID (MAC), now the device gets found. Also it reads the GATT service IDs (and the missed one is not listed). I uncomment the write to service (//pRemoteCharacteristic->writeValue(newValue.c_str(), newValue.length());) and the device keeps living.

Some finetuning...

Now I can search for an GAPP service (i.e. f000aa00-0451-4000-b000-000000000000), followed by Characteristic and write the value... nice :-)

Addons: 1) I had to add a delay(); in connectToServer() otherwise I get scrambled text output. Something seems to be wrong.

delay(50); /*--D- -(-6-293--)- -B-LE-U--t-i-l-s : coRennceecivtTeodS ae rvGeAPr  -e-ve--n-t-: --E-S-P-_-GA--P-_-B-LE--_-S-C-A-N-_S--TO--P-_C-O--M-P-L-E-T-E-_E--VT-<\r>-<\n>
-D-- -(6--3-0-7<\r>)<\n>

2) advertisedDevice.getApperance()); // Appearance ?

3)E (11925) BT: bta_gattc_cmpl_cback unknown conn_id = 3, ignore data<\r> Is this OK?

4) in loop() if (connected) { ... we should only write to the char if we really really sure that the char (and service) was found/is existing (or the function handle this)!!

5) got this sometimes:

V (9522) FreeRTOS: Semaphore released: name: SearchCmplEvt (0x3ffd6504), owner: getServices<\r><\n>
assertion "heap != NULL && "free() target pointer is outside heap areas"" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idD (9530) BLEClient: << getServices<\r><\n>
f-public/components/heap/./heap_caps.c", line 171, function: heap_caps_free<\r><\n>
abort() was called at PC 0x4015bedf on core 0<\r><\n>
<\r><\n>
Backtrace: 0x400887fc:0x3ffde060 0x400888fb:0x3ffde080 0x4015bedf:0x3ffde0a0 0x400839bb:0x3ffde0d0 0x40083e2d:0x3ffde0f0 0x4000bec7:0x3ffde110 0x400d9ac1:0x3ffde130 0x400d9b01:0x3ffde150 0x400d9c58:0x3ffde170 0x400d9cc4:0x3ffde190 0x400d9cd5:0x3ffde1b0 0x400dca3c:0x3ffde1d0 0x400d9ef4:0x3ffde1f0 0x400da661:0x3ffde210 0x400da679:0x3ffde230 0x400d823b:0x3ffde250 0x400d82d6:0x3ffde340 0x400d46a2:0x3ffde380 0x400d4a2d:0x3ffde410 0x400f4441:0x3ffde460 0x400f13b0:0x3ffde4e0<\r><\n>
<\r><\n>
Rebooting...<\r><\n>

6) in line 88: Serial.print("Found our device! address: "); there is no following address outout ? 7) Serial.print(advertisedDevice.getManufacturerData().c_str()); isn't working ?

cu Peter

LOG:

Editor: Moved to Pastebin: https://pastebin.com/dsTrrdQ7

SOURCE:

Editor: Moved to Pastebin: https://pastebin.com/gRFS9fED

[nkolban]

Howdy Peter, Am at work just now but will study the report in detail this evening ... however the immediate thing I wanted to examine was the source of the "BLE_client.ino" Arduino source file. The trace we were examining wasn't possible against the latest source. For example, the trace showed that the function called "connectToServer" was detecting an error but yet the code following that was still running. In the current source in Github, we check that connectToServer() succeeds before continuing. Here is the latest source file in the Github repository:

https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/tests/BLETests/Arduino/BLE_client/BLE_client.ino

Can I ask you to try this source file and make any changes you need/desire to this? Create a new Arduino project rather than replace your own source ... that way you can refer to both as you work. Look especially at the function called connectToServer(). Notice that it returns a bool return code while the copy in your code does not.

Looking at the Git file history, the change was committed on 2017-09-07.

[ghost]

Hi - no freeze with this code. :-) It also didn't find the service UUID, but this is well known - see the "Update:" in my last post. And - my code was from the zip file (see my 2nt post) - I didn't aware the newer one, sorry.
Peter

Editor: Moved to Pastebin: https://pastebin.com/frzpG0yN

[ghost]

OT: I play the the API and loop through the services, characteristics and values. I miss the return of the m_characteristicMap reference in getCharacteristics() similar as in the getServices()

std::map<std::string, BLERemoteService> BLEClient::getServices() void BLERemoteService::getCharacteristics()

I enhanced it to: std::map<std::string, BLERemoteCharacteristic> BLERemoteService::getCharacteristics()

Peter

[nkolban]

Looking at your trace, I see that your BLE partner device returns the following list of services:

00001800-0000-1000-8000-00805f9b34fb
00001801-0000-1000-8000-00805f9b34fb
0000180a-0000-1000-8000-00805f9b34fb
f000aa00-0451-4000-b000-000000000000
f000aa20-0451-4000-b000-000000000000
f000aa40-0451-4000-b000-000000000000
f000aa80-0451-4000-b000-000000000000
f000aa70-0451-4000-b000-000000000000
0000ffe0-0000-1000-8000-00805f9b34fb
f000aa64-0451-4000-b000-000000000000
f000ac00-0451-4000-b000-000000000000
f000ccc0-0451-4000-b000-000000000000
f000ffc0-0451-4000-b000-000000000000

You are looking for a service called:

0000aa80-0000-1000-8000-00805f9b34fb

Can you post a link to the actual peer device you are using? What makes you think that it exposes a service called 0000aa80-0000-1000-8000-00805f9b34fb? Do you have a data sheet or other documentation?

[ghost]

Yes, that's the point - unconnected (GAP) it provides only one UUID - different from all others in the GATT (connected). I don't know why there is in the GAP a service UUID at all.

In the example I use the well known 'Texas Instruments Sensor TAG', but the same happens with all other tested devices on my table (iTAG a kind of Keyfinder - more as an beacon, also connectable, Flowercare - a humidity plant sensor, ...).

http://www.ti.com/tool/cc2650stk www.dx.com/p/418518 https://www.open-homeautomation.com/de/2016/08/23/reverse-engineering-the-mi-plant-sensor/

Peter

[nkolban]

I have just ordered a couple of CC2650STK devices. Will await their arrival for testing.

[ghost]

;)

Keep in mind - the TI TAg is not in my focus, it's only a test object. As a hint - power the device externaly, the battery keeps only hours. There is a pin-connector behind a plastic plate

[ghost]

How can I access the remote descriptor and the handle of the remote characteristic ? Is this implemented yet? Peter

[nkolban]

Let us review the story. We have an ESP32 as a BLE client and a "device" as the remote BLE Server.

A BLE device exposes one or more services. A BLE service exposes one or more characteristics. A BLE characteristic exposes 0 or more descriptors.

The current implementation of the BLE C++ library for ESP32 does not yet remote device BLE descriptor access ... i.e. When the ESP32 is being a client, we can't yet access descriptors of a characteristics of a service. While that is not technically hard to do ... I have yet to find a device that actually exposes a descriptor of a characteristic ... and as such I am "demand driven" on what gets attention for implementation. Do you know that you need to access a descriptor of a characteristics of a service of a remote BLE server device? If so, can you help me understand what that device exposes in that fashion?

On the subject of handles, Internally, BLE implementations use "handles" to refer to instances of characteristics and descriptors. These then manifest themselves at API levels so that when you want to work with (for example), a characteristic instance, you need to know its handle. The intent of the C++ wrapper library (as opposed to the underlying raw ESP32 BLE APIs) is to attempt to make things simpler. To that end, concepts like "handles" are hidden from you. Instead, you can get access to a class instance that "represents" the logical characteristic ... for example ... which is easier to work with:

An instance of a class of type BLERemoteCharacteristic with all its accessor methods or the integer value "0x2a" which might represent a handle to a remote characteristic where you feed that handle value into many other APIs in order to maintain state.

So in your last question ... there were two concepts ... remote BLE descriptors (not yet implemented but keen to see a need/use case) and low level handles to characteristics (modeled out of the C++ libraries).

[ghost]

Right. One comment first. I'm not a expert ;) I work on this to lern how it works.

BTLE is Low Energy because one part is sleeping the most time, the other only listening. The sleeping part is often a sensor (Temperature, Hearttracker, ...), battery powered and called Peripheral, the listening part is the Central or Observer. (At this moment we don't have a connection and we don't speak about Server or Client). We use the GAP (here the A=Access). If the Central likes to connect the Peripheral it has to wait (!) for a (weakup and) Broadcast (or Adverticement). Depending the advertivement type (more then 7 different), the Central is able to

• only see (i.e. beacon),
• response (i.e ACK),
• response and request (short exchange),
• ...
• connect (long data exchance with GATT) (here the A=Attribute - rearly stupid)
• ... the Peripheral.

Only in the GATT mode we can talk about Server (provide data) and Client (get data). Not every time the Peripheral is the Server. And a device can be both (Server and Client) at the same time in one connection. This maybe change the organisation of the API and also the naming of the Examples of ESP-IDF.

So yes: A BLE >Server< exposes one or more services. A BLE service exposes one or more characteristics. A BLE characteristic exposes 0 or more descriptors. ... To find a device that exposes a Descriptor - the TI Sensor TAG does this as an example. As a hint if the tool is not known - I check all the things with the Nordic Android App 'nRF Connect' (blue icon with a 2 pieces chain).

Hmm, I can't post pictures here ?!

The descriptor describe a characteristic value with datatype (~unit) and string "Key press state".

i.e. with string [Service] Handle=0x005d Type=0xffe0 [Characteristic] Handle=0x005e ValueHandle=0x005f Type=0xffe1 Properties=(Notify) [Descriptor] Handle=0x0061 Type=0x2901(User Description) [Value] Key Press State

or with properties 'readable', could be important :) [Service] Handle=0x0010 Type=0x180a(Device Information) [Characteristic] Handle=0x0011 ValueHandle=0x0012 Type=0x2a23(System ID) Properties=(Read) [Value] [FF306400004C99B4]

more: https://developer.microsoft.com/en-us/windows/iot/samples/blegattdump

For me it's interessting to scan all possible things, I try to unterstand BLE and this is the practical part. The standard documentation has 2400 pages ;) Maybe my first ESP32 sketch should be a BLE scanner.

handles: They are interessting, because they are also used in the communication to reduce the need to transfere the compleate UUID. So, if you sniff a packet i.e with wireshark you only see the handle. Also often in the documentation or on webpages, in other tools and programms only the handles are printed.

hope it helps, its my view, no guaranty Peter

[nkolban]

Action: Wait for delivery of TI SensorTag device for testing same technology as issue raiser.

[nkolban]

I have received a TI SensorTag.

[nkolban]

I have the test and got working the TI SensorTag ... we we do a summary reset on this issue? Since we both now have the SensorTag, lets see if we can't get to grips with a usage scenario we can both test. Unfortunately Arduino-ESP32 with BLE C++ is currently broken while we wait on a fix for a toolchain issue ... but in the interim, if you still have a working Arduino environment because you haven't upgraded for a while ... lets pick just one of the puzzles and take it apart. Let me know which one you would like to look at.

[ghost]

Hello Neil. On my side I'm also working on this topic - more in the use and my unterstanding... :-) Yesterday I had a lot of problems and so I deceide to update all to the newst version (IDF and C++ API). I had do some mods to add my changes but it's compiling and some of my problems seems solved. I try yet to scan (discover) all from a peripheral (starting with the GATT of a server) and I get the following output now (Humidity sensor):

I also want to add the values and descriptors. It would be great so get your support at this point. Similar as at getServices I like to have the same methods for Characteristics (done) , Descriptors and Values. Also it would be nice to print the different UUID lenght types and the handles. If it is working well mybee you like to add it to your examples.

Back to topic: I would say the crash is solved and it was not the main problem. More: " It's a good way to compare the UUID of a Service in GAP to identify 'my' device? Alternatives: Device UUID (MAC) or Name. It's IMO more commen, easy to understand and short to type in. there is in my unterstanding no guaranty that a Service in the GAP is also existing in the GAPP.

Peter

[nkolban]

It sounds like the immediate puzzle is solved ... maybe we can close out this issue? Remember, we can always create new ones or re-open an old one if needed.

The browser based Tree View is interesting me though. There is a project called "ESP32_Explorer" (see https://github.com/nkolban/ESP32_Explorer) and it is being augmented to provide BLE visualization and editing. Would that also be useful to your tests?

[skyler14]

Hi Neil,

Is this related to the error I'm attaching below. Tried to add BLE support to another system about an hour ago and got this stacktrace:

Editor: Posted to pastebin: https://pastebin.com/WJzKu3QR

[chegewara]

Hi, ive got the same errors. To be sure, is it on arduino with old toolchain and newest Kolban's cpp_utils?

[nkolban]

Unfortunately this is related to #82

We are awaiting toolchain fixes from Espressif which should be available in a couple of weeks.

[ghost]

Hi Neil, here a bug summery of this thread and more:

1) Typo: advertisedDevice.getApperance()); // Appearance ? 2) Change the Example to find a device by mac or name, not by serivce ID 3) Improve the writeValue() to self check if the Characteristic exists and if the device is connected. 4) BLEUtils.cpp place the large Strings (members,services) into the PROGMEN, move the Structs into seperate files. 5) Enhance the getCharacteristics() to also return a pointer to the m_characteristic map like getServices() 6) Add the getDescriptors() , ... finctionality 7) Add a gethandle function (no use in other calls needed, only to print out informal) 8) Order of #include and #include "BLEDevice.h" (working), (BLEDevice WiFi - not working - 'WiFiServer' does not name a type)

If you agree, I can create seperate issues for each ? Peter

[nkolban]

Howdy my friend ... I'm a big fan of separate trackable issues per task ... that allows us to not miss somethign and makes these more readable ... however ... I do NOT want to put the burden of work on you (my customer/users). As such, you decide. My preference would be separate issues for each one ...

Weekend is coming so many hours to work on issues.

[ghost]

As you want ;)

Btw: the Editor looks fine ! Great to assess the ESP32 before using in own projects. Here is the state of the BLE Scanner:

[nkolban]

The editor does look great. Is this part of a project you will be releasing?

[ghost]

If you like you can provide it as one of your examples or integrate it into your ESP32 Editor. But in the moment, a lot of problems has to be solved :( 1) BLE things 2) BLE + Webserver work together 3) Upload data to Filesystem 4) ... 5) get enough coffee

[chegewara]

@pit001 Any progress with this project?

[ghost]

HI - I have learned that I have to investigate a lot of time (21 days) to realize BLE things with the yet version of API. To solve my actual functional need I switch to RPI with Bluez and Python and get it running in 2 days (and stud in vacation for 3 weeks ;) ) . Here I will continue if Wi-Fi and BLE will clean work together in the Arduino enviroment.

[chegewara]

I think it is possible with optimizations. I didnt check it with arduino, but ESP32Explorer is working on esp-idf. Since its pretty big app, and we made some optimizations to free ram which was hell issue, when you feel you are ready you can try.

Ive been asking because i like your services tree and been trying to do similar with jstree: https://github.com/nkolban/ESP32_Explorer/issues/7#issuecomment-341195453