v1.9.2-19 not discovering all peripherals

[lordthorzonus]

Hi 👋 and thanks for maintaining this fork! I updated my project for -19 from -15 release and noticed that some of my ble advertisements aren't discovered anymore by noble. Meaning I'm just not seeing them from:

noble.on("discover", (peripheral: Peripheral) => {
   logger.debug("Received BLE advertisement %s", peripheral);
});

Downgrading to -15 fixes this.

These peripherals in question are Xiaomi miflora devices (plant sensors), all my ruuvitags (temperature sensors from a different brand) are still discovered as usual so I'm lead to believe it's something to do with those advertisements particular. On the surface level though I cannot see anything special how these devices advertise:

{"address":"c4:7c:8d:6e:07:cf","state":"disconnected","services":null,"advertisement":{"localName":"Flower care","serviceData":[{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,247,207,7,110,141,124,196,13,8,16,1,49]}}],"serviceUuids":["fe95"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}}

{"address":"c4:7c:8d:6e:06:9c","state":"disconnected","services":null,"advertisement":{"localName":"Flower care","serviceData":[{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,170,156,6,110,141,124,196,13,4,16,2,235,0]}}],"serviceUuids":["fe95"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}}

Some other background info:

• I'm running on rasperry pi 3b+ with an external tplink dongle
• node 18.14.1

I'm happy to debug deeper, I'm not just so familiar where to start 😅. I noticed there was a lot of changes between -15 and -19 so I could not on a quick look figure out anything meaningful from the changes either. Thanks a lot!

[donavanbecker]

@Apollon77, have you noticed this at all? I have users reporting also.

[Apollon77]

I hear that the first time. I more hot Reports that Servicedata fills up. Ore and more over time and so used cpu more and more.

Maybe the idea is to Use debug logs ('DEBUG=noble,hci,att,gap node ....' and generate it for both Versions and then compare. Maybe we see something.

[lordthorzonus]

Hmm the issue you mentioned could be related. It seems to happen with these Miflora devices:

From -19 version:

2023-02-21T19:04:40.381Z hci onSocketData: 043e29020100009c066e8d7cc41d020106030295fe151695fe71209800179c066e8d7cc40d071003730000b3
2023-02-21T19:04:40.381Z hci    event type = 4
2023-02-21T19:04:40.382Z hci    sub event type = 62
2023-02-21T19:04:40.382Z hci        LE meta event type = 2
2023-02-21T19:04:40.382Z hci        LE meta event data length = 41
2023-02-21T19:04:40.382Z hci        LE meta event num reports = 1
2023-02-21T19:04:40.382Z hci        LE meta event data = 00009c066e8d7cc41d020106030295fe151695fe71209800179c066e8d7cc40d071003730000b3
2023-02-21T19:04:40.383Z hci            type = 0
2023-02-21T19:04:40.383Z hci            address = c4:7c:8d:6e:06:9c
2023-02-21T19:04:40.383Z hci            address type = public
2023-02-21T19:04:40.383Z hci            eir = 020106030295fe151695fe71209800179c066e8d7cc40d071003730000
2023-02-21T19:04:40.383Z hci            rssi = -77
2023-02-21T19:04:40.386Z gap advertisement = {"serviceData":[{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}}],"serviceUuids":["fe95"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}

And the service data slowly duplicates to this:

2023-02-21T19:06:01.380Z hci onSocketData: 043e19020104009c066e8d7cc40d0c09466c6f7765722063617265b1
2023-02-21T19:06:01.380Z hci    event type = 4
2023-02-21T19:06:01.380Z hci    sub event type = 62
2023-02-21T19:06:01.380Z hci        LE meta event type = 2
2023-02-21T19:06:01.381Z hci        LE meta event data length = 25
2023-02-21T19:06:01.381Z hci        LE meta event num reports = 1
2023-02-21T19:06:01.381Z hci        LE meta event data = 04009c066e8d7cc40d0c09466c6f7765722063617265b1
2023-02-21T19:06:01.381Z hci            type = 4
2023-02-21T19:06:01.381Z hci            address = c4:7c:8d:6e:06:9c
2023-02-21T19:06:01.381Z hci            address type = public
2023-02-21T19:06:01.381Z hci            eir = 0c09466c6f7765722063617265
2023-02-21T19:06:01.381Z hci            rssi = -79
2023-02-21T19:06:01.383Z gap advertisement = {"localName":"Flower care","serviceData":[{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,23,156,6,110,141,124,196,13,7,16,3,115,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,24,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,24,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,24,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,24,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,24,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,24,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,24,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,25,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,26,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,26,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,26,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,26,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,26,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,26,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,26,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,27,156,6,110,141,124,196,13,7,16,3,134,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,27,156,6,110,141,124,196,13,7,16,3,134,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,27,156,6,110,141,124,196,13,7,16,3,134,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,27,156,6,110,141,124,196,13,7,16,3,134,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,28,156,6,110,141,124,196,13,8,16,1,35]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,29,156,6,110,141,124,196,13,9,16,2,197,1]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,30,156,6,110,141,124,196,13,4,16,2,234,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,31,156,6,110,141,124,196,13,7,16,3,97,0,0]}},{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,31,156,6,110,141,124,196,13,7,16,3,97,0,0]}}],"serviceUuids":["fe95"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}

on -15 this is normal and looks like this:

2023-02-21T19:23:13.414Z hci onSocketData: 043e19020104009c066e8d7cc40d0c09466c6f7765722063617265ae
2023-02-21T19:23:13.414Z hci    event type = 4
2023-02-21T19:23:13.415Z hci    sub event type = 62
2023-02-21T19:23:13.415Z hci        LE meta event type = 2
2023-02-21T19:23:13.416Z hci        LE meta event status = 1
2023-02-21T19:23:13.416Z hci        LE meta event data = 04009c066e8d7cc40d0c09466c6f7765722063617265ae
2023-02-21T19:23:13.417Z hci            type = 4
2023-02-21T19:23:13.417Z hci            address = c4:7c:8d:6e:06:9c
2023-02-21T19:23:13.418Z hci            address type = public
2023-02-21T19:23:13.418Z hci            eir = 0c09466c6f7765722063617265
2023-02-21T19:23:13.418Z hci            rssi = -82
2023-02-21T19:23:13.419Z gap advertisement = {"localName":"Flower care","serviceData":[{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,134,156,6,110,141,124,196,13,4,16,2,235,0]}}],"serviceUuids":["fe95"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}

2023-02-21T19:23:20.455Z hci onSocketData: 043e29020100009c066e8d7cc41d020106030295fe151695fe71209800879c066e8d7cc40d071003e10000ad
2023-02-21T19:23:20.455Z hci    event type = 4
2023-02-21T19:23:20.455Z hci    sub event type = 62
2023-02-21T19:23:20.455Z hci        LE meta event type = 2
2023-02-21T19:23:20.455Z hci        LE meta event status = 1
2023-02-21T19:23:20.455Z hci        LE meta event data = 00009c066e8d7cc41d020106030295fe151695fe71209800879c066e8d7cc40d071003e10000ad
2023-02-21T19:23:20.455Z hci            type = 0
2023-02-21T19:23:20.455Z hci            address = c4:7c:8d:6e:06:9c
2023-02-21T19:23:20.455Z hci            address type = public
2023-02-21T19:23:20.456Z hci            eir = 020106030295fe151695fe71209800879c066e8d7cc40d071003e10000
2023-02-21T19:23:20.456Z hci            rssi = -83
2023-02-21T19:23:20.456Z gap advertisement = {"localName":"Flower care","serviceData":[{"uuid":"fe95","data":{"type":"Buffer","data":[113,32,152,0,135,156,6,110,141,124,196,13,7,16,3,225,0,0]}}],"serviceUuids":["fe95"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}

In my logic I'm always finding the service data with the uid like

const xiaomiService = peripheral.advertisement.serviceData.find((service) => service.uuid === XiaomiServiceId);

So i'm always getting the first one which is the same as the rest of the advertisements get infinitely appended to the serviceData array and not broadcasted as individual values as in -15 version 🤔

These sensors work in way that they broadcast multiple different advertisements always with the same serviceData uuid one at the time and in the data by parsing it one can figure out what type of sensor it is. So this seems to be the root issue 👍

[lordthorzonus]

Also why this does not happen with my ruuvitag devices is probably because they broadcast their values in the manufacturer data not service one:

2023-02-21T19:06:00.726Z gap advertisement = {"localName":"Ruuvi 8D5A","manufacturerData":{"type":"Buffer","data":[153,4,5,15,71,46,137,199,230,255,252,0,8,3,224,174,246,149,155,162,230,29,162,233,141,90]},"serviceData":[],"serviceUuids":["6e400001b5a3f393e0a9e50e24dcca9e"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}

2023-02-21T19:05:00.325Z gap advertisement = {"localName":"Ruuvi 8D5A","manufacturerData":{"type":"Buffer","data":[153,4,5,15,65,46,133,199,230,0,0,0,8,3,232,175,214,149,155,139,230,29,162,233,141,90]},"serviceData":[],"serviceUuids":["6e400001b5a3f393e0a9e50e24dcca9e"],"solicitationServiceUuids":[],"serviceSolicitationUuids":[]}

😅

[lordthorzonus]

@Apollon77 I think I managed to find the source of the problem https://github.com/abandonware/noble/pull/297

[Apollon77]

Cool. Will review as soon as Kids in bed. @AlCalzone maybe interesting!!

[Apollon77]

Ps: does that also fix your discovery issue?

[lordthorzonus]

Yeah 👍 My own discovery logic broke as the new service data just got appended to the array as my code expected it to be unique one entry by id 😁

[lordthorzonus]

solved in #297