search

cph-cachet / flutter-plugins

A collection of Flutter plugins developed by CACHET
551 stars 681 forks source link

[Health 10.2.0] Fetch Awake, REM, Core, and Deep attributes as seen in Apple Health #985

Closed pkhivesara closed 2 months ago

pkhivesara commented 5 months ago

Fetch Awake, REM, Core, and Deep attributes as seen in Apple Health

Device / Emulator and OS

Please complete the following information for each phone and/or emulator you're experiencing this bug on:

Describe the bug

How can we obtain sleep_rem, sleep_core, sleep_deep, and sleep_awake, time_asleep, and time_in_bed entries from this health package, similar to the sleep intervals shown in the Apple Health app?

image

I do see a closed ticket here: https://github.com/cph-cachet/flutter-plugins/issues/803

However it appears we do not get the correct data based on the thread in the above ticket.

DominicOrga commented 5 months ago

I could obtain the Awake, REM, Deep and In Bed data collected from my Apple Watch. However, the SLEEP_CORE is missing.

IMG_5920

khaleelkhalifa commented 4 months ago

I second this For myself It seems that Asleep unspecified and Sleep core is missing which was explained in #803

I investigated and figured out the problem here.

Apple allows the following Sleep Types (Reference: Health Kit Documentation) :

  1. inBed : The user is in bed
  2. awake: The user is awake
  3. asleepCore: The user is in light or intermediate sleep
  4. asleepDeep: The user is in deep sleep
  5. asleepREM: The user is in REM sleep
  6. asleepUnspecified: The user is asleep, but the specific stage isn't known. (Possibly captured when the watch isn't used for sleep detection and sleep is logged manually or automatically based on phone usage) Note: asleep is now deprecated.

Currently, the mapping for some of these in this package is incorrect. Refer this file - https://github.com/cph-cachet/flutter-plugins/blob/master/packages/health/ios/Classes/SwiftHealthPlugin.swift.

Currently, SLEEP_IN_BED - 0 SLEEP_ASLEEP - 1 SLEEP_AWAKE - 2 SLEEP_DEEP - 3 SLEEP_REM - 4

How it should be (corrections in bold) - SLEEP_IN_BED - 0 SLEEP_ASLEEP - 1 (sleepUnspecified in terms of HealthKit. Value is going to be zero when all sleep stages' data is available.) SLEEP_AWAKE - 2 SLEEP_LIGHT - 3 (sleepCore in terms of HealthKit. This should be allowed as a data type similar to Android.) SLEEP_DEEP - 4 SLEEP_REM - 5

I am going to attempt this change and will raise a PR if that works. EDIT - Noticed that a PR is already here: #816 . Though this PR doesn't deal with SLEEP_LIGHT to be added as a datatype which is a crucial data point.

SlimShadyIAm commented 3 months ago

Hi all,

Thanks for raising this issue. When I try to query the data requested in the original issue using these types:

  static final types = [
    HealthDataType.SLEEP_AWAKE,
    HealthDataType.SLEEP_ASLEEP_CORE,
    HealthDataType.SLEEP_ASLEEP_REM,
    HealthDataType.SLEEP_ASLEEP_DEEP
  ];

I am able to get results for all of the requested types. A screenshot from my Apple Health app: image

... compared to a quick visualization I made using the datapoints that the health package returns:

image

the order of the y-axis in the plot isn't correct but the data seems to align nevertheless, and I am able to get datapoints for core sleep as well. If there's something I'm misunderstanding could you please explain further?

pkhivesara commented 3 months ago

@SlimShadyIAm Can you share a code snippet for your example? I think our issue is that the data is not consistent compared to what we see on apple. And SLEEP_CORE does not pull any data at all?

khaleelkhalifa commented 3 months ago

Hi, The one I am missing is AsleepUnspecified. https://developer.apple.com/documentation/healthkit/hkcategoryvaluesleepanalysis/asleepunspecified

I believe apple made that category when the stages were implemented and deprecated the old option.

Note, It seems that the categories that you mentioned above are not listed in the readme docs

SlimShadyIAm commented 3 months ago

@pkhivesara I used the example app which you can find in our repo.

On these lines: https://github.com/cph-cachet/flutter-plugins/blob/310d13d8f91b6481b82389a8f558c1010678b5c5/packages/health/example/lib/main.dart#L40-L58

you can set types to:

  static final types = [
    HealthDataType.SLEEP_AWAKE,
    HealthDataType.SLEEP_ASLEEP_CORE,
    HealthDataType.SLEEP_ASLEEP_REM,
    HealthDataType.SLEEP_ASLEEP_DEEP
  ];
SlimShadyIAm commented 3 months ago

Hi, The one I am missing is AsleepUnspecified. https://developer.apple.com/documentation/healthkit/hkcategoryvaluesleepanalysis/asleepunspecified

I believe apple made that category when the stages were implemented and deprecated the old option.

Note, It seems that the categories that you mentioned above are not listed in the readme docs

Yes, I can see that this is unclear. I'm going to look into what needs to be changed here -- it also seems like we have multiple sleep related HealthDataType categories that seem to serve the same purpose, and also how to support the unspecified sleep type.

SlimShadyIAm commented 3 months ago

And SLEEP_CORE does not pull any data at all?

@pkhivesara just to be clear you are using HealthDataType.SLEEP_ASLEEP_CORE?

SlimShadyIAm commented 3 months ago

I've looked into how sleep stages have previously been implemented in the health package and noticed a few things with regards to what is documented vs what is supported.

This table shows the current state of supported sleep stages. I noticed some of the types are overlapping and it was unclear which was supported on what platform.

Stage iOS Android Notes
SLEEP_IN_BED ✅
SLEEP_ASLEEP ✅ ✅
SLEEP_AWAKE ✅ ✅
SLEEP_DEEP ✅ ✅
SLEEP_LIGHT ✅
SLEEP_REM ✅ ✅
SLEEP_OUT_OF_BED ✅
SLEEP_SESSION ✅
SLEEP_ASLEEP_CORE ✅ This was undocumented. Rename to SLEEP_CORE for consistency?
SLEEP_ASLEEP_DEEP ✅ This was undocumented. Serves the same purpose as SLEEP_DEEP, combine
SLEEP_ASLEEP_REM ✅ This was undocumented. Serves the same purpose as SLEEP_REM, combine

This is how I propose it should be changed. While it might not solve a potential issue that @pkhivesara is raising, it would provide clarity for the types and documentation, improve consistency and add support for all sleep stages across both HealthKit and Health Connect.

Stage iOS Android Notes
SLEEP_UNKNOWN ✅ ✅ (NEW) on iOS this is SLEEP_UNSPECIFIED, name changed to fit Android (consistent with other measures)
SLEEP_AWAKE ✅ ✅
SLEEP_CORE ✅ renamed for consistency
SLEEP_LIGHT ✅
SLEEP_DEEP ✅ ✅ combined
SLEEP_REM ✅ ✅ combined
SLEEP_IN_BED ✅
SLEEP_SLEEPING ✅ (NEW)
SLEEP_OUT_OF_BED ✅
SLEEP_AWAKE_IN_BED ✅ (NEW)
~SLEEP_ASLEEP~ ~✅~ Remove? This value currently simply corresponds to core sleep: link to code

I did look into the suggestion from the previous issue linked above (https://github.com/cph-cachet/flutter-plugins/issues/803#issuecomment-1804712277), however I'm unsure whether SLEEP_LIGHT and SLEEP_CORE describe the same stages of sleep - ChatGPT says they don't. And also I don't think SLEEP_ASLEEP should correspond to asleepUnspecified in HealthKit terms because HealthKit already provides us with a category value for this.

I'd like to hear your thoughts on this before I make this change, if you have any differing suggestions please let me know.

khaleelkhalifa commented 3 months ago

I am happy with that, It will add the missing stage which our app needs. How long would you say you need for this to go out? Thanks Khaleel

SlimShadyIAm commented 3 months ago

How long would you say you need for this to go

We're planning a major release in the next couple of weeks and it could potentially be included in that. Would you be able to help test whether getting unspecified sleep works? I looked through my own personal data but couldn't find any recordings which had unspecified sleep.

khaleelkhalifa commented 3 months ago

Hi, Sure that is not a problem on my end. The unspecified stage seems to be linked to certain ios watch software versions.

You can also manully add stages of sleep via the apple health app if that helps. It is what I have been doing to test and debugg the issue we were getting.

Thanks Khaleel

pkhivesara commented 3 months ago

SLEEP_ASLEEP_CORE

Nope, this attribute is not listed on the docs from what I see. You are suggesting that SLEEP_ASLEEP_CORE indeed returns the correct values. Is that right? I will give it a shot and drop an update here once done.

SlimShadyIAm commented 3 months ago

Yes, this isn't in the README but please give it a try.

SlimShadyIAm commented 3 months ago

On second thought, Apple's documentation says asleepCore also refers to light sleep, so I will combine ASLEEP_CORE into ASLEEP_LIGHT and add a note in the docs that this also refers to core sleep.

Also, Apple's asleepUnspecified is defined as:

The user is asleep, but the specific stage isn’t known

which seems to correspond to STAGE_TYPE_SLEEPING on Android:

The user is asleep but the particular stage of sleep (light, deep or REM) is unknown.

so I will also combine these into SLEEP_ASLEEP.

As a result, SLEEP_UNKNOWN will become Android only, representing STAGE_TYPE_UNKNOWN: https://developer.android.com/reference/kotlin/androidx/health/connect/client/records/SleepSessionRecord

Stage iOS Android Notes
SLEEP_ASLEEP ✅ ✅ (NEW) - on iOS this refers to asleepUnspecified
SLEEP_AWAKE ✅ ✅
SLEEP_AWAKE_IN_BED ✅ (NEW)
SLEEP_DEEP ✅ ✅ combined
SLEEP_IN_BED ✅
SLEEP_LIGHT ✅ ✅ combined - on iOS this refers to asleepCore
SLEEP_OUT_OF_BED ✅
SLEEP_REM ✅ ✅ combined
SLEEP_UNKNOWN ✅ (NEW)
pkhivesara commented 3 months ago

Yes, this isn't in the README but please give it a try.

Sorry forgot to reply yesterday, but asleep_core seems to point to asleep_light as per Apple, which is what my original question was here. How does one pull in the CORE sleep data?

SlimShadyIAm commented 3 months ago

Thanks @pkhivesara but I'm having some trouble understanding the issue you're facing --- reading SLEEP_ASLEEP_CORE in version 10.2.0 points to HKCategoryValueSleepAnalysis.asleepCore as you can see here in our Swift implementation of getData:

https://github.com/cph-cachet/flutter-plugins/blob/310d13d8f91b6481b82389a8f558c1010678b5c5/packages/health/ios/Classes/SwiftHealthPlugin.swift#L849-L851

There isn't a separate "light sleep" category on iOS, asleepCore covers that according to Apple's docs. We have no way of differentiating the two and we can only read the data HealthKit provides as it is.

In my earlier comments, I queried for all the sleep stages the Health package currently supports and the visualization was the same as the one in the Health app, including core sleep.

Could you please help me understand if I'm missing something or clarify where you're seeing asleep_core associated with asleep_light?

SlimShadyIAm commented 3 months ago

I've just opened #1026 to implement the changes discussed above - would greatly appreciate if you all could help test. If you have Android devices with sleep data on Health Connect, it would also be greatly appreciated if you could test that.

@pkhivesara, please check if you are able to access the data you require using this version.

pkhivesara commented 2 months ago

I've just opened #1026 to implement the changes discussed above - would greatly appreciate if you all could help test. If you have Android devices with sleep data on Health Connect, it would also be greatly appreciated if you could test that.

@pkhivesara, please check if you are able to access the data you require using this version.

We tried the changes you suggested, but we are seeing few issues with the latest library update.

If you see the screenshots below, the asleep interval times fetched from the plugin does not match the hours shown in Apple health. Is that a known issue?

image image

SlimShadyIAm commented 2 months ago

Hi,

That seems strange, but it's not clear from the information you provided - is this a sleep session you added manually to the Health app and you chose the "Asleep" category?

I'm assuming yes; I tried to reproduce this by adding a session like so:

image

Then I used the example app to query for the data and passed in this as the types:

  static final types = [
    HealthDataType.SLEEP_ASLEEP,
  ];

And I got this as the result:

image

SlimShadyIAm commented 2 months ago

I've merged the PR to address the original issue in this thread, as before it was not clear how to query sleep data on iOS. If you are still experiencing problems could you please open a new issue?