Bleacon occasionally stops reporting beacons with two Bluetooth devices installed

[jfathman]

I ran this beacon scan on RPi 3 successfully for 11 days:

var Bleacon = require('bleacon');
Bleacon.startScanning();
Bleacon.on('discover', function (bleacon) {
    console.log(Date(), bleacon.uuid, bleacon.major, bleacon.minor,
        bleacon.measuredPower, bleacon.rssi, bleacon.proximity, bleacon.accuracy);
});

That test used the integrated Bluetooth hardware in RPi 3 (device hci0).

I installed the iogear GBU521 Bluetooth USB adapter (device hci1) and ran the same test above with NOBLE_HCI_DEVICE_ID=1. Bleacon reads beacons from the GBU521, but stops reporting beacons after a few minutes (beacon source is nearby, from another RPi3 that provided beacons for the 11 day test above).

If I Ctrl-C the test app and restart it, Bleacon reports beacons again, but eventually suspends.

I added code to stopScanning and startScanning if there are no discovered beacons for 5 seconds:

var Bleacon = require('bleacon');
Bleacon.startScanning();
var count = 1;
var timer = null;
Bleacon.on('discover', function (bleacon) {
    clearTimeout(timer);
    console.log(Date(), count, bleacon.uuid, bleacon.major, bleacon.minor,
        bleacon.measuredPower, bleacon.rssi, bleacon.proximity, bleacon.accuracy);
    timer = setTimeout(function () {
        console.log('TIMED OUT: count =', count);
        Bleacon.stopScanning();
        count++;
        Bleacon.startScanning();
    }, 5000);
});

That keeps the app running, and beacons are reported except for occasional 5 second quiet periods until the timer fires. I can see in the count logged to the console this happens perhaps 25 times in half a day or so.

If I run two instances of the app simultaneously, one using RPi 3 integrated Bluetooth and another using the GBU521, both experience the timeouts that are recoverable.

Since it ran for 11 days before installing the GBU521, there seems to be some issue that surfaces when two Bluetooth receivers are active.

I can live with the restart logic, but wanted to mention this in case it is of interest.

For the latter test above with restart logic, I am running:

• Raspberry Pi 3
• Raspian GNU/Linux 8 (jessie)
• Linux kernel 4.1.19-v7+
• bluez 5.23-2+rpi1
• Node.js 6.1.0
• bleacon 0.4.0
• iogear GBU521

Really appreciate this excellent open source Bluetooth software for Node.js.

[sandeepmistry]

@jfathman thanks for the heads on this and the positive feedback!

It would be interesting to compare this behaviour to sudo hcitool lescan --duplicates to see if it's an adapter issue or something to do with noble + it's dependencies.

[jfathman]

@sandeepmistry Good idea, it may be next weekend before I can try that. I switched from bleacon to noble several days ago, and have not noticed anything since, so there could be an eventing difference. Now that I have more experience with both libraries, I might be able to spot something (including something I may have been doing wrong) next time. I'll follow up. May be a few days. I've been running noble on 10 Raspberry Pi 3's with an external Bluetooth adapter GBU521 installed in addition the RPi 3 internal Bluetooth adapter, and it works great.

[bsturgess]

Just to let you know I am having exactly the same issue. With both HCI0 (onboard bluetooth on pi3), and HCI1 (usb csr 4.0 dongle).

Unrelated (i think), but also the reading rate (number of reads per minute of a beacon at fixed 1s broadcast output) is much higher when using USB dongle. I think perhaps the UART rate on pi3 for btle chip is low?

[jfathman]

Correction to my earlier report --

With no external Bluetooth USB adapter installed and only the RPi 3 integrated Bluetooth adapter, the latter is hci0 as you would expect.

When I installed the iogear GBU521 Bluetooth USB adapter, I assumed it would appear as hci1. But it does not. It (USB) becomes hci0 and the integrated Bluetooth (UART) becomes hci1:

[jfathman@rpi-02 ~]$ hciconfig
hci1:   Type: BR/EDR  Bus: UART
        BD Address: B8:27:EB:34:A2:29  ACL MTU: 1021:8  SCO MTU: 64:1
        UP RUNNING 
        RX bytes:162285376 acl:0 sco:0 events:4033968 errors:0
        TX bytes:3278864 acl:0 sco:0 commands:495874 errors:0

hci0:   Type: BR/EDR  Bus: USB
        BD Address: 5C:F3:70:62:68:2A  ACL MTU: 1021:8  SCO MTU: 64:1
        UP RUNNING 
        RX bytes:37818 acl:0 sco:0 events:1373 errors:0
        TX bytes:2484 acl:0 sco:0 commands:124 errors:0

Sorry for the misinformation in my initial report. I am retesting now and will see if I can duplicate the earlier problem and explore with sudo hcitool lescan --duplicates as suggested by @sandeepmistry.

[jfathman]

I also now see what @bsturgess observed, where far fewer beacons are reported by the RPi 3 integrated Bluetooth adapter via UART than are reported (for the exact same beacon source) by the external USB Bluetooth adapter. We might suspect that the integrated Bluetooth adapter discards beacons if there is not room in a limited buffer in front of the UART to queue up an entire beacon just received.

If so, that is a really important observation by @bsturgess because it means we might be compelled to use an external Bluetooth adapter (added cost) in lieu of the integrated adapter. The two adapters are not interchangeable, but have very different throughput capabilities in a world where ambient beacon traffic is ever increasing.

[bsturgess]

Yes I ran some tests overnight (12hr period), and with a CSR 4.0 type usb adaptor I received 4x the beacon detections than the onboard received (total of 40,000 vs 10,000), and had 300 incidents of more than 10 seconds between reads (requiring bleacon restart). Whereas the USB had 0 incidents of such 'blackouts'.

Do you think its buffer or baud rate related?

I have now scrubbed the use of onboard bluetooth and will soley use USB adaptors. Only real advantage pi3 now gives us in our application is onboard wifi.

And thanks for this thread, I thought I was going mad!

For the test: 3 Beacons were at -8db and a broadcast interval of 950ms. (estimote beacon) So this still shows a difference to actual broadcasted pings which in theory should have been roughly 130,000 ((12 * 3600) * 3)

[jfathman]

Here is a new observation that surprised me.

Running the Node app (source further above) to read beacons from the RPi 3 integrated Bluetooth adapter (UART), the beacon read rate is low at ~3 per second as we have been discussing:

[jfathman@rpi-02 node]$ sudo NOBLE_HCI_DEVICE_ID=1 node app.js | pv -l -i 3 -r >/dev/null 
[3.02 /s]

But when I read that same UART based adapter using hcitool -i hci1 lescan --duplicates, the beacon read rate is ~12 per second:

[jfathman@rpi-02 ~]$ sudo stdbuf -o0 hcitool -i hci1 lescan --duplicates | pv -l -i 3 -r >/dev/null 
[12.4 /s]

That ~12 per second rate is more like what we were seeing when bleacon read the external USB Bluetooth adapter.

I would have expected the USB or UART interface to be transparent to bleacon/noble, abstracted behind some kind of hci# device open, with the beacon rates reported by hcitool lescan to be about the same.

[bsturgess]

That is interesting. Watching this thread closely.

[jfathman]

It will probably be next weekend before I can spend time on this again. If we could figure out why bleacon/noble is reporting beacons at only 1/4 the rate as hcitool -i lescan --duplicates and achieve parity, then perhaps the RPi integrated Bluetooth adapter could get back in the game @bsturgess and we wouldn't necessarily need the external Bluetooth adapter with added cost (might still want it for better antenna characteristics, but at least we would have a choice.

I ran some bleacon/noble experiments with DEBUG=noble to see if there were any events reported by noble that were perhaps not handled when running bleacon, but at the point where the beacon reporting stalled, noble did not log any events.

[sandeepmistry]

@jfathman @bsturgess have you had a chance to compare the scan parameter noble uses vs hcitool -i lescan --duplicate?

sudo hcidump -t -x will let you see what is being sent the to HCI adapter.

These can be modified here: https://github.com/sandeepmistry/noble/blob/master/lib/hci-socket/hci.js#L227-L246

[jfathman]

Thanks @sandeepmistry, I was traveling this weekend and did not have access to my test bench, but I remain interested in this exploration and will return to it perhaps next weekend.

I continue to operate 10 RPi 3's running noble with the idle restart logic, and it has been running without a hitch for several weeks now.

I agree that the hci0/hci1 assignment depends on the kernel device detection logic, and we might conclude that Linux enumerates USB connected devices before UART (or Bluetooth via UART) connected devices. The shifting of the integrated Bluetooth adapter from hci0 to hci1 is surprising at first, but once recognized is deterministic. I have considered adding a little hciconfig output parsing at startup just to confirm the device I specify is in fact 'USB' or 'UART' as expected, but that would be just to confirm which device is actually being used to catch any oversight in my application configuration.

[vdharmon]

I had a similar issue running Bleno and Noble on two ble devices. One on the UART and one on the USB. I was trying to pin Bleno to the USB and Noble to the UART. For my noble code I wasn't getting the Noble to pin on hci1 (UART). It wasn't the Noble code, but an issue I had with my code. I exposed a property this._deviceId in my code locally thru the hci.js file so I can determine which hci port my code is running on. I was then able tell, the Noble code was also going to hci0 and crashing Bleno.

It would be great if you could add that on your next round of updates for both bleno and noble. This would help jfathman with some of his testing above also.

sandeepmistry: I'm very impressed with both the Noble and Bleno code. It's worked quite well for me. Thanks for your hard work.

[sandeepmistry]

@vdharmon thanks for the positive feedback!

I exposed a property this._deviceId in my code locally thru the hci.js file so I can determine which hci port my code is running on.

Would you mind sharing you patch for this. The deviceId property would be available on Linux only (not applicable to OS X).

[vdharmon]

Sure. I've attached a zip file that has the hci.js files for both Bleno and Noble. The hci.js file is prefixed with bleno or noble to distinguish which package each goes with.

hci.zip

[sandeepmistry]

@vdharmon thanks, could you submit a pull request for those changes please.

[vdharmon]

Pull requests created for both Noble and Bleno.

Regards, Vince

[sandeepmistry]

@vdharmon great! I've made a few comments, please let me know your thoughts in the pull requests.

[sandeepmistry]

Any updates on this? Is it still a problem?

[vdharmon]

I'm performing more testing to make sure there's no issue. There might be, but I want to track it down first. If I find there is I'll get you all the details.

[vdharmon]

The changes you updated are perfect.

Thanks

[sandeepmistry]

Can we close this issue then?

[sandeepmistry]

Closing this for now, please let me know if there are still issues.