Web Demo not working?

[danperks]

Using latest Chrome, on Windows, I cannot get the web demo to work.

The device is found in the pairing stage, and appears to pair successfully. However, after this point, nothing else happens.

Chrome Console Log (Blank line between each entry):

THREE.WebGLRenderer 90

OBJLoader: 194.5078125ms

Uncaught (in promise) DOMException: GATT operation failed for unknown reason.

ControllerBluetoothInterface.js:160 Error: NetworkError: Failed to execute 'writeValue' on 'BluetoothRemoteGATTCharacteristic': GATT Server is disconnected. Cannot perform GATT operations. (Re)connect first with `device.gatt.connect`.

Any steps I've missed, or has a chrome update changed something?

[mtvv]

Same here – I also get the error message mentioned above. Additionally I get the following:

Uncaught RangeError: byte length of Int32Array should be a multiple of 4
    at new Int32Array (<anonymous>)
    at ControllerBluetoothInterface.onNotificationReceived (<anonymous>:98:30)

Anything more we can do to help debug the issue?

[DanEdens]

Still having Same issue. I'm going to start digging into this https://developer.android.com/reference/android/bluetooth/BluetoothGattServer

[andikay]

I am also encountering the same issue as the thread starter. I am wondering where to add "device.gatt.connect" in the code to get it to work again.

[Venryx]

There appear to be two hurdles involved here.

The first is this:

Uncaught (in promise) DOMException: GATT operation failed for unknown reason.

A variant of this seems to be the error:

Uncaught (in promise) DOMException: GATT Error Unknown.

I get the variant version, and it is triggered by this line:

this.customServiceNotify.startNotifications();

I'm not sure what the cause is, but I've tinkered with the code for about an hour, and haven't been able to reliably get past it. My fork can be seen here (cleaned up a bit, and partially converted to TypeScript): https://github.com/Venryx/gearvr-controller-webbluetooth

I got past the error once (getting what appeared to actually be data in the onNotificationReceived function), but haven't been able to reproduce since then. That one time I got past it, was after having left my computer (on) for about an hour, coming back, and immediately trying to pair; somehow, it magically paired then, but not since.

It's also worth noting that even without the startNotifications() line above, the gatt-server backing the connection disconnects like 100ms later anyway. So these observations make me think that perhaps a firmware update occurred for newer versions of the device, which makes it more finicky with third-party access (perhaps requiring a certain command to be sent immediately on connection).

Anyway, this brings us to hurdle 2: The second error encountered by @mtvv (https://github.com/jsyang/gearvr-controller-webbluetooth/issues/8#issuecomment-676069710):

Uncaught RangeError: byte length of Int32Array should be a multiple of 4
    at new Int32Array (<anonymous>)
    at ControllerBluetoothInterface.onNotificationReceived (<anonymous>:98:30)

The one time that I got past hurdle 1, I hit the second error shown above. Looking at the code, however, it appears that this is just a trivial error due to the code grabbing the first three bytes from the buffer, rather than the first four (four bytes are required for a valid int32 value). I changed the code on that line to new Int32Array(buffer.slice(0, 4))[0], and I'm pretty sure this will fix the issue. (if not, it would just require another trivial change, like adding a 0 byte or the like)

Unfortunately, I can't confirm the fix for the second issue, since I haven't been able to get past issue 1 since then at all.

Anyway, because of this, for now I'm giving up on the web-bluetooth approach, and trying a lower-level access to the device through noble, possible using node-web-bluetooth on top for a more familiar interface. Noble is apparently an alternative "Bluetooth stack", bypassing the Windows Bluetooth stack -- which I'm hoping will mean I can connect to it without "issue 1" above being a blocker.

[Venryx]

For context, I am aware of five repositories where the author has successfully connected the GearVR controller to custom code: (will update this post if I find more) 1) https://github.com/jsyang/gearvr-controller-webbluetooth (this repo) 2) https://github.com/sameer/gearvr-controller-uhid 3) https://github.com/rdady/gear-vr-controller-linux 4) https://github.com/gb2111/Access-GearVR-Controller-from-PC 5) https://github.com/ilyabru/GearVrController4Windows

In trying to implement NodeJS access to the device through noble, I will be referencing the repos above.

It's worth noting that according to this comment, the gear-vr-controller-linux repo also seems to be hitting a disconnect issue (similar to issue one described above, but perhaps with a longer delay of 10s, rather than <100ms).

It's thus possible that all of the solutions above have a similar disconnection/unknown-error issue to that described in my previous post. (let's hope not; I should contact some of those repo owners to ask if they've also hit disconnection issues with their devices/libraries in recent months)

[andikay]

I have come further by now. Both GATT errors are related to the device not being able to connect properly. What you need to do there is turn off and back on Bluetooth on your computer, then remove controller from the new created list. Then you can reload the page and go through the motions of connecting/pairing the controller and start data updates.

The issue you will then encounter is the one at the "const timestamp" with "byte length of Int32Array should be a multiple of 4 at new Int32Array". Setting the slice from "0, 3" to "0, 4" makes the demo work for about 1-2 seconds and then the error appears again.

I have commented out all timestamp stuff for testing and what happens then is that the controller shows the rotation and buttons accurately for those 1-2 seconds and then vanishes off the screen :D.

I have debugged this demo for a day until I figured out how this works and that ArrayBuffer is not a normal array and slicing an ArrayBuffer is not getting the indices but the bytes, which leads to vastly different results. I needed this for a project of mine in C# and it took a lot of debugging to understand what's happening. This also prompted me to update the functions to more useful versions. Like the developer said in his reverse engineering article: never assume that the work of someone else is correct :).

[Venryx]

@andikay You saved us! :D

I had given up on the web-bluetooth approach, but your two observations above put me over the hurdle. The web demo is now working completely for me!

Regarding:

the controller shows the rotation and buttons accurately for those 1-2 seconds and then vanishes off the screen :D

The fix for that is just to run the "Start controller data updates" command again.

I inferred this from the comment at the bottom of the ControllerDisplay.js file: // Have to do the SENSOR -> VR -> SENSOR cycle a few times to ensure it runs

After running that twice, the controller stays connected for as long as I keep the page open.

(Update: I just updated the demo to perform this double-calling process automatically; it works on the first run now.)

Anyway, the fixed demo can be seen in my fork here.

Reminder: Follow the instructions for the demo (based on @andikay's comment above), as it still requires the bluetooth-reconnect process each time you pair.

[andikay]

Cool stuff! But are you sure that slicing eventData gives the same results as slicing the buffer? Because when I did that the results were completely whack. The reason for this is that the ArrayBuffer can display the incoming data in a few different ways, depending on what kind of array you want to put the sliced data (or bytes, rather) into. In my debugging sessions the ArrayBuffer displayed the bytes as Int8Array (60 entries), Uint8Array (60 entries; equals eventData), Int16Array (30 entries) and Int32Array (15 entries). When I sliced the eventData Uint8Array the values I got were completely different from when the slice happened on the ArrayBuffer into an Int32Array.

edit I just saw that you changed it back to the ArrayBuffer 👍 .

The fix for that is just to run the "Start controller data updates" command again.

This does not work for me at all. when I press it again nothing happens (the controller is still gone and the console does not print anything). But your demo works very well for me! Now that I was able to try the demo properly I noticed that the orientation of the controller does not appear to set itself to the correct starting position. The user has to hold the controller like on the screen and then start it, but the rotations are not 1:1 the same. I would love to have it working "flawlessly" like in this video.

Do you have experience with the Madgwick AHRS algorithm that is used in the demo ( https://www.npmjs.com/package/ahrs )? I use the C# version of it ( https://github.com/xioTechnologies/Open-Source-AHRS-With-x-IMU/ ) and that one does not take any timestamp or deltaTime as parameter.

[Venryx]

I just saw that you changed it back to the ArrayBuffer 👍 .

Yeah, I made that mistake of slicing eventData at first, but noticed something was wrong when the device only rotated like 1% of the actual movement. :)

Now that I was able to try the demo properly I noticed that the orientation of the controller does not appear to set itself to the correct starting position.

Yes, it appears to not have a robust "absolute alignment" system. I think the Madgwick library used is supposed to be able to perform this absolute alignment (using Magnetometer data -- presumably the magX, magY, and magZ fields in the controller's data), but it doesn't appear to work (at least not reliably). Perhaps the data was parsed incorrectly from the bytes, or was passed incorrectly.

As for the AHRS algorithm, no, I don't have experience with it. However, looking at the source of the JS version and the C# version, it appears they do both have a deltaTime parameter. (called deltaTimeSec/recipSampleFreq/recipNorm in the JS one, and SamplePeriod in the C# one)

[andikay]

As for the AHRS algorithm, no, I don't have experience with it. However, looking at the source of the JS version and the C# version, it appears they do both have a deltaTime parameter. (called deltaTimeSec/recipSampleFreq/recipNorm in the JS one, and SamplePeriod in the C# one)

The SamplePeriod is an argument you pass when you initiate the algorithm. With the npm package, this is also done at initiation, see here: https://github.com/jsyang/gearvr-controller-webbluetooth/blob/master/ControllerDisplay.js#L14 <- This is not related to the deltaTime.

[Venryx]

@andikay Ah okay; I don't know what it's for then. Let us know if you find out, and/or how to stabilize/correct the inferred controller rotation.

I thought I'd also mention that I managed to recreate the GearVR controller connection code using the NodeJS noble Bluetooth stack: https://gist.github.com/Venryx/a3fd1e2ca2fe7a4c87c9e4e630c48ec5

Once connected, it works exactly the same as the web-bluetooth api. (which sits atop the standard Windows bluetooth stack)

Overall I prefer this new approach, because although: 1) Drawback: It requires that the WinUSB generic driver be installed for the Bluetooth adapter. 2) Benefit: It avoids the need for the obnoxious disableBT->enableBT->unpair process each time the user wants to connect! (something is messed up in the Windows Bluetooth stack apparently)

Of course, using noble is only possible if you're writing a NodeJS/Electron app (not a mere website). But if your use-case permits, I recommend it for programs that will frequently be connecting to the device.

(Though, I suppose an alternative is to keep using the web-bluetooth api, but automate the disableBT->enableBT->unpair process somehow, so that at least the user doesn't need to do the clicking themselves.)

[andikay]

I believe that the disableBT->reenableBT->unpair is related to the controller itself and not the driver. Let's not forget, this controller was not meant to be used outside of the Samsung GearVR application.

[Venryx]

I believe that the disableBT->reenableBT->unpair is related to the controller itself and not the driver.

If it's related to the controller itself (and not the driver/bluetooth-stack), then why is it that since switching to the "noble" bluetooth-stack, I have been able to connect and reconnect dozens of times without a single disableBT, reenableBT, or unpair operation required?

(Granted, I am currently refreshing the page between connection attempts, as I haven't worked out the code to disconnect the previous connection yet; but refreshing the page is easier than those three external steps, and doesn't seem like it would be equivalent, unless noble is shutting down [and clearing pairing for] the whole stack per refresh. I guess that's possible, but either way, the current stack is much easier to use for new connections.)

Note also that: Not only does the connection work every time without that process, it's also significantly faster -- the delay between clicking Pair and it connecting is like one-third of the web-bluetooth's delay (which utilizes the Windows bluetooth stack).

[andikay]

i can tell you that I currently use a Unity version of this framework ( https://www.btframework.com/ ) which has its own DLL and what happens there is that after pairing, connecting and reading all the services and characteristics the controller automatically disconnects with an 0x00030008 error, which is "The connection has been closed." This only does not happen when I do the disableBT->reenableBT->unpair routine. This BT framework also disconnects automatically when you close the connection.

[Venryx]

Interesting.

Have you confirmed that the Bluetooth connection through that DLL works, even with Bluetooth disabled in the Windows settings? (as this would confirm that it's not using parts of the Windows bluetooth stack behind the scenes)

[andikay]

I cannot recall if I initially tried it without having BT in Windows turned on, but I will try it the next time I work on it (shouldn't be more than a few days) and will share the results here.

[andikay]

I just checked, my project does not work when Windows BT is turned off. It then says "no BT radio found".

[andikay]

@andikay Ah okay; I don't know what it's for then. Let us know if you find out, and/or how to stabilize/correct the inferred controller rotation.

I have made a bit of progress by now. Research as well as trial & error showed that the initialization of the Madgwick AHRS algorithm (aka the "SamplePeriod") is very relevant to how the rotation values are displayed. When setting this float value closer to 1, the controller rotation display becomes more and more "agitated" and shaky and overreactes to controller movement. When setting the value closer to 0, the controller rotation display becomes more and more "lazy" and does not display controller rotation properly anymore.

From the German support website of the Gear VR Headset ( https://www.samsung.com/de/support/mobile-devices/leistungsmerkmale-und-kompatibilitat-der-samsung-gear-vr-with-controller-sm-r325/ ) I found the information that the controller has a sampling rate of 208 Hz (meaning 208x per second) when in the 0800 VR Mode. Therefore, I put the SamplePeriod to "1f / 208f", which is ~0,0048. In comparison, the creator of the web demo put it to "1f / 68.84681583453657", which is ~0,014525 and makes the controller display too jittery.

The downside of my value is that some movements are lagging behind a bit. Maybe there needs to be a multiplier value applied or some fine tuning is in order. I am not done testing yet :).

In addition, I had better success of stabilizing the controller rotation display when not using the magnetometer at all, meaning I use the Madgwick AHRS update function without the magnetometer values (see here for the usage of the npm package used in the web demo: https://www.npmjs.com/package/ahrs#functions ).

[cwig]

This is an interesting discussion. I have been doing some of my own exploration of the Gear VR controller and have made some progress on getting the magnetometer to work with the sensor fusion. The main issue is that the magnetometer seems to be in the touchpad area of the controller and is at about a 30 degree offset from the accelerometer and gyroscope. Also, the magnetometer needs to be calibrated.

I have a github pages up with this fix https://cwig.github.io/gearvr-exploration/index.html. After you successfully connect the controller, if you hold down the “back” and “trigger” button and swing the controller in a figure 8, it will calibrate the magnetometer. Then the sensor fusion should be much better. Maybe some of the other ideas in this discussion could be applied as well to make the fusion even better. I’ve been exploring more things with the controller such as applying firmware upgrades that you can read about in the README of the repo https://github.com/cwig/gearvr-exploration if you are interested.

[andikay]

That's a great discovery that you made @cwig. I will test your demo soon to see if I can reproduce your results. Getting rid of the drift is the biggest challenge I have at the moment.

The drift can mainly be witnessed on the Y-axis and it rotates the controller clockwise. I created some logic to counter this drift which involves a timer (value increases constantly) and a constant value that is multiplied by the timer. The product of this calculation is then subtracted from the Y-axis. I use eulerAngles for this as trying to do this with quaternions just lets the controller go crazy (also, quaternions can only be multiplied in Unity, which is where I use the controller). Unfortunately, different persons have different results with this technique. While I do not have drift with this on different machines and smartphones (I deploy a prototype application on Android), others do have counter-clockwise drift with it.

I have also adjusted my values for the Madgwick algorithm by now to use the SamplePeriod suggested by the creator of this repo, but I use a Beta of 0.01 instead of 0.352 in order to have smoother movement without the controller being so jittery.