search

polarofficial / polar-ble-sdk

Repository includes SDK and code examples. More info https://polar.com/en/developers
Other
474 stars 153 forks source link

Receiving HR=0 that is Almost Certainly Invalid and Other Problems #197

Closed KennethEvans closed 1 year ago

KennethEvans commented 3 years ago

Platform on which you observed the bug:

Device on which you observed the bug:

Describe the bug Near the time the first SDK was released I wrote a demo to compare HR and RR for two devices. At one time it was part of the SDK examples. I have not used it for some time, but recently got a Sense and tried comparing it to the OH1 I have been using. There are several problems.

  1. There are long periods where the HR returned from the API is zero. (This is based on LOG messages, not any plotting, but the plotting seems to be accurate.) "Long" means several seconds, usually not more than 10. I do not believe this is accurate. A screenshot is attached. I am recalling sequences of HR = 0 happened with the original app after it was first connected, but not after it had "settled down". I assumed it was valid and an artifact of the OH1. Now it is continuing to have periods of HR = 0 after long periods of time. I now suspect it was not ever an artifact but a bug.

HR = 0 in both the hrNotificationReceived data and the PPI data at about the same time. It seems to be consistent in this respect.

Note: It is not obvious, but the line starting with Tot is shows the total of the accumulated RR values compared to the total elapsed time, in case you are curious. They tend to be the same for the OH1 and Sense. This is not true for some other PPG devices. ;-) It was added for debugging and has not been removed.

When using Beat, as I do regularly, the HR is never 0 (but maybe they are suppressing zeros).

I have my own BLE app that uses the BLE Heart Spec and works with any device that supports it, which is most of them. It uses BLE directly and does use the SDK. It does not show any HR = 0. I know HR = 0 is NOT suppressed. Attached is a screenshot for 5 min with the Sense. It has no zeros in the 5 minutes. I believe this is the correct output from the device.

  1. The status light is blinking magenta (might be considered purple) while it is connected with the SDK. It does not do this when connected with Beat or my app. This is on both the OH1 and the Sense. I do not know what this means.

  2. The load for this app is very light. The data come in at about 1 Hz compared to 130 Hz for ECG. I do not believe it is a result of CPU overload.

How to Reproduce This project is on GitHub at https://github.com/KennethEvans/KE.Net-HR-Compare. I am comparing an OH1 and a Sense. It happens every time and with both devices. Since this problem started, I have converted the app to using SDK 3.2.1 instead of 1.1.0. The behavior is the same before and after I converted, but it is not good to submit a bug using some old version. The current version in the repository uses 3.2.1.

Expected behavior HR should seldom, if ever, be zero.

Screenshots and logs If applicable, add screenshots or logs to help explain your problem Screenshot_20210919-133027_KENet HR Compare Screenshot_20210919-135647_BLE Cardiac Monitor

JOikarinen commented 3 years ago

Hi @KennethEvans,

the reason for 0 HR is that when enabling the PPI stream the firmware switches on different heart rate calculation algorithm. The reason why algorithm is changed is that Pulse to Pulse interval (PPI) calculation is much more sensitive for any movements of the device or other possible interference. Algorithm calculates primary the PPI and then from the PPI the heart rate in bpm. That heart rate is then delivered with PPI stream and heart rate service. I believe the consequences of that algorithm switch is why you see problems when using PPI stream via SDK and why you not seeing problems with the Beat app.

KennethEvans commented 3 years ago

Thank you for the information. I assume the reason for the blinking magenta or purple light is related to the algorithm change.

The problem is that this algorithm is apparently wrong. It is unlikely my HR is 0 at all and even more so for several seconds. I now consider the output from my compare program to be worthless for PPG devices. Data that lie are worse than no data.

There seem to be problems calculating PPI as mentioned in #194. I don't understand the details at this time. You cannot calculate HR without knowing PPI, and the HR will be no more accurate than the PPI it is based on.

I would be concerned that the HR is even accurate in the OH1 and Sense. However, when HR is not zero, the numbers appear to agree, and I have seen at least one study covering multiple devices, presumably using the usual BLE Heart spec, that indicate good agreement with the H10. (I trust the H10.) The problem seems to be with this internal algorithm.

Erhannis commented 2 years ago

I, too, experience this bug. It occurs on both Android and iOS, btw. When you e.g. move your arm around more than slightly, the PPI has "blockerBit" true (but is still present), and HRs are 0 (for several PPI after the movement). You say "Pulse to Pulse interval (PPI) calculation is much more sensitive for any movements of the device", but if that were so, then shouldn't PPI be the values to disappear, rather than HR? We're receiving the values that are sensitive to motion, but not the ones that shouldn't be. The thresholds are so sensitive that we get almost no heart rate values at all.

JOikarinen commented 2 years ago

@Erhannis good questions. Here are some comments, which hopeful help with your development:

It occurs on both Android and iOS, btw.

  • the SDK doesn't touch the values received from the sensor so both platforms behave the same as you noticed When you e.g. move your arm around more than slightly, the PPI has "blockerBit" true (but is still present)
  • this was the design choice still let the PPI value available, but by blockerBit indicating its quality You say "Pulse to Pulse interval (PPI) calculation is much more sensitive for any movements of the device", but if that were so, then shouldn't PPI be the values to disappear, rather than HR?
  • HR is zero as it is not thought as reliable anymore The thresholds are so sensitive that we get almost no heart rate values at all.
  • yes, that's the main problem with PPI calculation, it is sensitive for the motion
KennethEvans commented 2 years ago

I will repeat what I have said before: HR for all these devices, ECG or PPG, is not measured directly, it is calculated from measuring intervals.

The Sense appears to give a HR consistent with the H10 and other brands. Thus, in its normal operating mode it is using some PPI that is more accurate than what is coming from the SDK.

It would be nice if that PPI were made available. (I am assuming here the problem is not in the SDK.)

In addition the HR in normal operating mode is not 0. It would be a real problem if that HR were not thought to be reliable.

KennethEvans commented 2 years ago

I did some more studies with a Verity Sense.

Not only is it giving HR = 0, it is giving random values that are not the same as reported by the H10. Waving my arm does seem to cause zeros, but am also getting these zeros and random values sitting down and not moving much.

I turned off using PPI entirely (does nothing in streamingFeaturesReady). It is still giving random HR that are different from the H10. (Image below.) The purple light is on. When the purple light is on, my other, conventional BLE app is also showing these screwy values and zeros. Stopping the SDK app does not turn off the purple light. (I am calling api.shutdown in onDestroy.) I had to turn the Sense off and then on for the light to go away. After doing that I get sensible values in the other app.

I am not sure what is happening with the purple light and what mode it is getting into, but it is giving incorrect results when using the SDK whether using PPI or not.

Screenshot_20211021-134415_KENet HR Compare

JOikarinen commented 2 years ago

@KennethEvans thanks again for good points and analysis. I collect your comments and try to make good arguments on those.

I will repeat what I have said before: HR for all these devices, ECG or PPG, is not measured directly, it is calculated from measuring intervals.

The Sense appears to give a HR consistent with the H10 and other brands. Thus, in its normal operating mode it is using some PPI that is more accurate than what is coming from the SDK.

It would be nice if that PPI were made available. (I am assuming here the problem is not in the SDK.)

  • what you say is exactly how I was originally thinking. Now since you I have risen the issue, I have had conversions with the people in Polar who have done the PPI and HR algorithms (those are separate algorithms as discussed before). Both algorithms takes the PPG as input, but generates a bit different output, PPI algorithm does it best to detect PPI whereas HR algorithm does it best to output reliable HR. In your previous comment, the statement : HR for all these devices, ECG or PPG, is not measured directly, it is calculated from measuring intervals. is what I thought too. What I have learned is that there are alternative ways as well with PPG signal. From your point of view I understand the frustration as I am not able to open up the discussion any further, rest of details are proprietary.

Stopping the SDK app does not turn off the purple light.

  • PPI streaming is not stopped in this case. Purple light is indicating PPI measurement being on.
KennethEvans commented 2 years ago

Thanks for the information. It is interesting the HR is not calculated from intervals. In any event it is wrong when using PPI in the SDK and seems to be right otherwise. One wonders how accurate the fitness test is.

Also it was not right in the example above that was not starting PPI. It was still requesting it as a feature, though.

I know at least one PPG device that does supply what appear to be reasonable intervals in the RR field. This is the Corsense, which is designed for HRV analysis. Elite HRV has a whole business model that depends on it. Thus, it can be done.

How do you turn off the purple light with the SDK? That is, apparently, how do you stop the PPI? You want to do this in onDestroy. It is a problem to have the device be left in this mode. I have not been watching the purple light as it is under clothing. I do know sometimes the device will not connect again, and at times it gives an exception, saying it is already in that mode.

JOikarinen commented 2 years ago

How do you turn off the purple light with the SDK?

  • purple light is indicating the PPI measurement, so stopping the PPI stream will set device back to normal heart rate measurement mode. Few alternatives:
  • you may dispose started PPI stream
  • alternatively when the device is disconnected the PPI measurement stops:
  • you may call api shutdown, as a consequence the connection is closed to the device and stream is closed
  • you may call api disconnect

I do know sometimes the device will not connect again, and at times it gives an exception, saying it is already in that mode.

  • the exception is thrown if the device has the requested stream on. If I understood your situation correctly, the correct exception is thrown (i.e. PPI stream was really on), but you expected the PPI stream to be closed already?
KennethEvans commented 2 years ago

I am calling api shutdown in onDestroy. I have in addition tried api disconnect in a try catch (since it might not be connected), but that is currently commented out. It didn't help.

but you expected the PPI stream to be closed already?

Yes. It happens when the app is newly started. It should have been shutdown when the app previous exited. It appears api shutdown is not doing what it should. (Sometimes.)

I am pretty much using the same logic as in the demo. The code is available on Github as mentioned above. https://github.com/KennethEvans/KE.Net-HR-Compare

JOikarinen commented 2 years ago

Thanks for sharing your code, I tried it out and found a bug from SDK. The bug is that the PolarBleApiDefaultImpl.kt code is missing @JvmStatic annotations and as a consequence when defaultImplementation is called from Java code the result is two different instances of Polar BLE SDK api.

Furthermore, as the Polar BLE SDK api is intended to be singleton, I see that HRActivity has two instances of PolarBleApi created. So what you would need to do, is something like this in HRActivity:

// to have only single instance of SDK API
public PolarBleApi mApi;
...
mApi.setApiCallback(new PolarBleApiCallback() {
// here you need to combine what you have written to existing mApi1 and mApi2 callbacks
...
}
KennethEvans commented 2 years ago

Thanks so much for doing that. I can't get to it right now but will as soon as possible. At the moment (without the code in front of me) I'm not sure how to distinguish the two devices in the callbacks. It does seem to work with two api's (apart from the shutdown issue).

Maybe it doesn't have to be a singleton? If it does, it should be written to get the single instance and not allow instantiating multiple instances, I would think.

JOikarinen commented 2 years ago

At the moment (without the code in front of me) I'm not sure how to distinguish the two devices in the callbacks.

  • each of the callbacks are called with identifier (i.e. deviceId) to indicate which device is causing the the callback. So one instance of API is enough, you may just check on each callback which was the device causing the callback.

Maybe it doesn't have to be a singleton?

KennethEvans commented 2 years ago

If it does, it should be written to get the single instance and not allow instantiating multiple instances, I would think.

What I meant is that in cases of a singleton usually there is something like PolarBleApi.getInstance(). Then there is no way to have multiple instances (and hence no unpredicted problems 😉).

JOikarinen commented 2 years ago

Yep, the getInstance is hiding under defaultImplementation function.

JOikarinen commented 1 year ago

Closing as this issue is containing discussion on multiple different aspects of SDK, original problem described is reported internally at Polar