Error (and maybe workaround) connecting to BLE device in Unity and Oculus Quest 2

[JavaMcGee]

Hi there, Nice stuff and the educational material helps understand what's going on. I first tried the Unity classes and Java plugin (as-is) with a simple test app running on my phone (Pixel 7/Android 13) - that worked great to scan, connect and subscribe to a ESP32 device that sends back bike wheel rotation data (Cycling Speed and Cadence characteristic profile). However, using the same test app code to run in VR on a Quest 2 (Android 12) gets errors trying to connect, as shown in the following logcat:

• 12:08:08.234 BluetoothGatt connect() - device: 94:B9:7E:D5:16:B2, auto: true
• 12:08:08.234 BluetoothGatt registerApp()
• 12:08:08.234 BluetoothGatt registerApp() - UUID=f4037b99-4d2c-4c45-80a4-53efc3faa7c0
• 12:08:08.236 bluetooth system/bt/stack/gatt/gatt_api.cc:1019 GATT_Register: Allocated name:GattClient uuid:aca2c142-9438-15a8-641d-17bed8faaac1 gatt_if:6 eatt_support:0
• 12:08:08.237 BluetoothGatt onClientRegistered() - status=0 clientIf=6
• 12:08:08.238 bt_btif_config system/bt/btif/src/btif_config.cc:137 btif_get_device_type: Device [94:b9:7e:d5:16:b2] device type 3
• 12:08:08.238 bt_stack [ERROR:gatt_api.cc(1171)] GATT_Connect: Unsupported transport for background connection gatt_if=6
• 12:08:08.238 bt_stack [ERROR:bta_gattc_act.cc(392)] bta_gattc_init_bk_conn unable to connect to remote bd_addr=94:b9:7e:d5:16:b2
• 12:08:08.238 BluetoothGatt onClientConnectionState() - status=133 clientIf=6 device=94:B9:7E:D5:16:B2
• 12:08:08.239 BluetoothGatt close()
• 12:08:08.239 BluetoothGatt unregisterApp() - mClientIf=6

Digging around Google for the meaning of an "unsupported transport for background connection" turned up only one conversation that suggested the Java call to connectGatt might need a fourth parameter to specifically set the transport type as BLUETOOTH_LE rather than a default on some Android 12 devices:

Original: device.connectGatt(UnityPlayer.currentActivity.getApplicationContext(), true, service.gattCallback);

Suggestion: device.connectGatt(UnityPlayer.currentActivity.getApplicationContext(), true, service.gattCallback, BluetoothDevice.TRANSPORT_LE); (also, Android 12 on Quest 2 no longer requires the LOCATION permissions on the manifest which are prohibited by Meta.)

Recompiling the Java plugin with this change does allow connections on the Quest 2, which seems good for this particular case. But I don't have experience with the Bluetooth API to be sure this is definitively the right way to fix this.

[Velorexe]

Hi @JavaMcGee ,

Great that you were able to build and use the Java library yourself! That really helps in the process. Since I haven't been able to test with the Oculus Quest 2 (was limited to Pico VR devices), this was something that I wasn't able to test myself. Same goes for Android 12 to be completely honest.

I'm thinking of doing a bit of a rewrite to make connecting to Android 12 and below more straight forward and easier, since it's honestly a mess at the moment.

Besides the fact that you're able to connect, are you also getting the data correctly and everything? If so it sounds like your problem is solved with your efforts.

And if that's the case, can I keep this issue open as a reminder to implement such a change when I do the rewrite?

[JavaMcGee]

I'm still testing the data side of things. I am able to subscribe and receive the expected data values parsed from the byte array message. Just that it's not entirely a realistic scenario because I'm connecting to an ESP32 to emulate what I hope behaves like a commercial BLE sensor with the cycle speed/cadence profile, and I still might have some issues on the device side.

You can certainly keep the issue open.

Another minor thing I noticed was the 'stop scan' seemed unresponsive at first but it turned out to just be the FinishedDiscovering message only comes back after the scan timeout period in the postDelayed thread, even though the scan actually stopped immediately. Wondering if the FinishedDiscovering message could be moved to the ScanCallback when mScanning is false, or something like that.

[izukingz]

@Velorexe, One note if you are considering a rewrite to support oculus quest2. Meta removed Bluetooth scan support from their sdk and you cannot submit an app in their store that includes that because of Location permissions. They instead require that devs use Companion Device API to connect to the device before BLE communication. There is a discussion here with a possible code snippet: https://communityforums.atmeta.com/t5/Quest-Development/Bluetooth-scan-and-permissions-BLE-and-the/m-p/971091/highlight/true#M4785

[Velorexe]

Thanks for the information @izukingz !

Though I gotta ugh at the changes that Meta is doing to the "open-ness" of their Android environment. I'm currently rewriting the library to be less about solely passing things to the BleManager, but having devices and characteristics manage their own connection / data instead.

One thing I could take a look at is either different Java libraries to accompany the C# side of things, or create a "godly"-esque library, which would create a whole different mess.

The good thing about having different Java libraries is that you only need to include the one that you'd like to use or the one that you have to use (say for example Android SDK levels), bad side about it is that you have to pay attention to which library gets compiled with your APK every time you compile to a certain Android platform.

Wondering what other people think about this idea, or if there are solutions that I'm not seeing.

[izukingz]

@Velorexe, I think different libraries will be better. Even more, different projects. One project will be a rewrite that you are currently doing. Another one will be the Companion Device API and BLE. Btw Android also discourages using Location permission unless your app really needs it.

[Velorexe]

So I'll treat this thread as the "Refactor / V2" thread for now, feel free to unsubscribe for now.

I've reworked the entire system to get rid of the commands, since the entire framework was already designed to be aimed towards "reacting" to events, instead of rapid firing actions to BLE from the get-go. So I've split up the BleManager and device instructions into a BleDevice class, with which you can interact directly to BleGattServices and BleGattCharacteristics. Most to all properties of these classes will be pulled from the moment you connect to a device, so you have an overview of the type of the characteristic, permissions, etc.

You can see an overview of it here: https://github.com/Velorexe/Unity-Android-Bluetooth-Low-Energy/blob/refactor/project-overhaul/Assets/Scripts/BLE/Device/BleDevice.cs

I also overhauled the java library to be easier to build (by including Unity's classes.jar and the entire Android Studio project with Gradle tasks) and to check for Permissions and such. The UnityAndroidBLE will not be returned to Unity's side if the permissions check fails. For now it works with Android 12 from what I've experienced, will have to test for Android 11 and lower soon.

You can see that over here: https://github.com/Velorexe/Unity-Android-Bluetooth-Low-Energy-Java-Library/blob/develop/app/src/main/java/com/velorexe/unityandroidble/UnityAndroidBLE.java

As per @JavaMcGee's suggestion, I've added a way to define your own Transport when connecting to the device and will default to BLUETOOTH_LE instead of BLUETOOTH_AUTO from now on. I'm aiming to complete the rewrite some where around this weekend.

[JavaMcGee]

Wow, you've been busy - you're efforts are appreciated and sounds like you've reworked it to cover many features people need.

A couple extra minor suggestions:

• In the BLEMessageAdapter, when you receive json messages from the plugin, parse the BleMessage and push that on a ConcurrentQueue, then have an Update function that dequeues the BleMessages and invokes the Unity events. This brings the events back to Unity's main thread where the callbacks are not on the main thread. [Oops, ignore that, it's not necessary as apparently Unity's SendMessage does it already]
• Optional arguments for a service uuid and device address passed to the plugin scan, used for a ScanFilter to limit the results to that specific device or service type. At least useful in my case to just find CSC devices and to quickly find a user's previously selected preferred device.

Since my post last week, I needed to keep momentum going and decided I need to understand the Android BLE API more thoroughly if I'm going to debug and maintain this key feature of my app. Also my use case is quite a bit simpler; no need for the parallel command queues and only need to connect to a single device of a specific type at one time, and only need to support the permission model for Android 12+. Also a few extra abilities like the scan filter mention earlier. So I started from the basics in the Android Developer guide for Bluetooth. Ended up with a relatively light-weight wrapper around the Java API calls and callbacks, and a different but similar opinion on the class design and events sent back to Unity. Initial tests on the Quests 2 seem to be working nicely.

Obviously the Java plugin is only going to work after a build and deploy to an Android/Quest device, not in the Editor or Quest Air Link. To keep working on my app and UI within the Unity Editor in Windows, I found it helpful to have an interface IBLEClient for the main functions of setup, scan, connect, enableData, etc. and two adapters implementing that, an AndroidBLEPluginAdapter and a WindowsBLEPluginAdapterMock. A BLEClientFactory chooses one based on the environment (UNITY_EDITOR_WIN or UNITY_ANDROID). The AndroidBLEPluginAdapter of course passes the calls through to the Java plugin, while the WindowsBLEPluginAdpaterMock simulates the Java plugin while developing in the editor.

[Velorexe]

Wow! I'm glad that you're going so deep into BLE and such, gives a good insight into new features I could add. Your suggestion for the filter is implemented as well!

You can find it here: https://github.com/Velorexe/Unity-Android-Bluetooth-Low-Energy/blob/refactor/project-overhaul/Assets/Scripts/BLE/BleManager.cs#L90

The idea for a mock sounds like a really good one, helps a lot while you're in the editor. I'm trying to think of ways to make this generic enough so that I can implement such a system in my plugin as well, but since data and such differs to much from project to project, it needs to be build in such a way that it could be applied to any project.

I'll think about it and if you have any suggestions, feel free to send them my way! I'll pass on credit to you as well of course.

Since my system is now also using taskId's to keep track of which task is doing what (and where to send updates from the Java library), I was thinking of making an Editor window that keeps track of the different tasks that are send out while your phone is connected. Since all communication is passed through the Android LogCat anyway (and Unity's Debug.Log is also send on that stack), you could parse all the data from there and show them inside an EditorWindow.

[Fedrogan]

Hi! I'm trying to connect to mi band 4 and I see this error in logcat 2023.09.07 00:30:41.282 18181 18208 Error Unity AndroidJavaException: java.lang.NullPointerException: Attempt to invoke virtual method 'android.bluetooth.BluetoothGattCharacteristic android.bluetooth.BluetoothGattService.getCharacteristic(java.util.UUID)' on a null object reference 2023.09.07 00:30:41.282 18181 18208 Error Unity java.lang.NullPointerException: Attempt to invoke virtual method 'android.bluetooth.BluetoothGattCharacteristic android.bluetooth.BluetoothGattService.getCharacteristic(java.util.UUID)' on a null object reference 2023.09.07 00:30:41.282 18181 18208 Error Unity at com.velorexe.unityandroidble.UnityAndroidBLE.readFromCharacteristic(UnityAndroidBLE.java:472) 2023.09.07 00:30:41.282 18181 18208 Error Unity at com.unity3d.player.UnityPlayer.nativeRender(Native Method) 2023.09.07 00:30:41.282 18181 18208 Error Unity at com.unity3d.player.UnityPlayer.-$$Nest$mnativeRender(Unknown Source:0) 2023.09.07 00:30:41.282 18181 18208 Error Unity at com.unity3d.player.UnityPlayer$C$a.handleMessage(Unknown Source:122) 2023.09.07 00:30:41.282 18181 18208 Error Unity at android.os.Handler.dispatchMessage(Handler.java:102) 2023.09.07 00:30:41.282 18181 18208 Error Unity at android.os.Looper.loopOnce(Looper.java:210) 2023.09.07 00:30:41.282 18181 18208 Error Unity at android.os.Looper.loop(Looper.java:299) 2023.09.07 00:30:41.282 18181 18208 Error Unity at com.unity3d.player.UnityPlayer$C.run(Unknown Source:24) 2023.09.07 00:30:41.282 18181 18208 Error Unity at UnityEngine.AndroidJNISafe.CheckException () [0x0008

How i can fix this?

UPD: Apparently the reason for the error is that I didn't do this: //Replace these Characteristics with YOUR device's characteristics

[izukingz]

@Velorexe, I'm wondering if you have gotten the chance to look into using CompanionDevice pairing for acquiring the device before connecting for BLE? (https://developer.android.com/guide/topics/connectivity/companion-device-pairing)

[Velorexe]

Hi @izukingz

I haven't taken a good look at it yet. From what I've gathered, it's a replacement for searching for the new Android 26 API? I've also heard that the new Quest devices are also using that to find new devices. I'm thinking of making a separate Java library just for the Quest / 26 API to make it easier to connect with them (or I'll incorporate it within a new version of the library).

[izukingz]

@Velorexe, yes it's a replacement for scanning for BLE device which requires Location permission. With this you don't need to scan, instead the companion device will give you the BLE device and you can go ahead and connect just as you are currently doing in your current library. They have a sample code in the link I sent above for Bluetooth classic. I think it's similar for BLE except to change the filter to BLE. Let me know if you work on it and need me to test it. Thanks