search

vicara-hq / kai-dotnet

The .NET client for using the Kai SDK.
https://vicara.co
4 stars 6 forks source link

Questions and clarifications #24

Open dumblob opened 5 years ago

dumblob commented 5 years ago

a) When and how will be haptics supported in SDKs?

b) How are quaternions in Kai computed? What are they useful for? Is it just combination of the 3 sensors (accelerometer, magnetometer, gyroscope)? Is it somehow smoothed (e.g. by floating average filter)?

c) How are PYR data computed? Is it just combination of the 3 sensors (accelerometer, magnetometer, gyroscope)? Is it somehow smoothed (e.g. by floating average filter)?

d) How can one calibrate sensors (especially accelerometer, magnetometer, gyroscope)? There seems to be no direct access, but because of their drift, calibration will be necessary (see Inertial Navigation Systems for issues - especially the Zero Velocity Update trick).

e) Can Kai provide proportional information about fingers? From FingerPositionalData it seems it can, but without knowing how this data is acquired, there is no way to find out whether this data is linear, what is the proper range, how to calibrate the data, etc.

f) What is the maximum frequency with which Kai can send accelerometer, magnetometer, and gyroscope data?

g) What is the latency to read out accelerometer, magnetometer, and gyroscope data? How to decrease the latency (it feels currently somewhat delayed)?

rakshith-ravi commented 4 years ago

Hello @dumblob. Thank you for reaching out to us. To answer your questions:

a) When and how will be haptics supported in SDKs?
The haptics control will be enabled in the next update. We will release an ETA soon. The haptics API will allow you to set the duration for the vibration upto a hardware controlled maximum. This limitation is in place as the haptic motor has a significant effect on battery life if not properly managed.

b) How are quaternions in Kai computed? What are they useful for? Is it just combination of the 3 sensors (accelerometer, magnetometer, gyroscope)? Is it somehow smoothed (e.g. by floating average filter)? The quaternions are computed directly from the Accelerometer and Gyroscope raw data. The magnetometer is not used for the time being due to high susceptibility to EMI. The quaternions are not not filtered.

c) How are PYR data computed? Is it just combination of the 3 sensors (accelerometer, magnetometer, gyroscope)? Is it somehow smoothed (e.g. by floating average filter)? The PYR is calculated by applying a transform to the quaternions data. The PYR data is not filtered either.

d) How can one calibrate sensors (especially accelerometer, magnetometer, gyroscope)? There seems to be no direct access, but because of their drift, calibration will be necessary (see Inertial Navigation Systems for issues - especially the Zero Velocity Update trick). There is an IMU calibration function provided that adjusts for the bias/offset that causes this drift. It averages 10 seconds of data to execute this. Additionally, the zero velocity update is not required as the Kai does not keep track of absolute spatial position. It uses relative motion and absolute orientation to execute its gesture recognition.

e) Can Kai provide proportional information about fingers? From FingerPositionalData it seems it can, but without knowing how this data is acquired, there is no way to find out whether this data is linear, what is the proper range, how to calibrate the data, etc. This version of the Kai will not be able to provide finger positional data. The function that you’re seeing is experimental and is currently quite buggy. If there is any progress on this front, we will inform you.

f) What is the maximum frequency with which Kai can send accelerometer, magnetometer, and gyroscope data? The Kai internally samples the sensors at 1KHz. The data is sent over Bluetooth every 9ms. The minimum for the BLE protocol is 7.5ms but it is currently set to 9ms for better battery life.

g) What is the latency to read out accelerometer, magnetometer, and gyroscope data? How to decrease the latency (it feels currently somewhat delayed)? Measurements that we’ve done maintain a consistent latency of <10ms total. Our next firmware update will further decrease this by around 1ms.

dumblob commented 4 years ago

Thanks for the very informative answer. Those are good news. Therefore one more question if you don't mind:

h) Does your mouse emulation (i.e. mouse cursor movement as used for playing FPS games) do anything to compensate the (mathematically speaking) derivative function characteristics of the sensor data? The sensor data deliver acceleration instead of velocity, but to move the cursor naturally, our brains expect the cursor to move according to velocity, not acceleration (this is also the reason why all such devices have the worst possible reputation for cursor movement). Do you perform any integration or similar compensation of the sensor data?

Of course, plain integration wouldn't probably suffice due to noisy data and thus unacceptable delay when integrating, but this could be compensated by many means (from basic ones like Kalman filter estimation through fuzzy logic filtering up to neural networks filtering).