[Feature request] Readiness and Activity data

leikoilja commented 2 years ago

Would it be possible to also fetch readiness data such as:

readiness score
Heart Rate Variability (average)

nitobuendia commented 2 years ago

Hi @leikoilja - thanks a lot for the feature request.

If the data is available within the sleep endpoint, I am happy to add it as it requires very low effort. However, I believe none of these are included there.

Readiness score is available in the API (doc). However, it would require involving a new endpoint. This adds complexity as you would need to either keep different sensors or merge them somehow. One brings complexity to the configuration. The other brings complexity to the merging (e.g. what happens if there's sleep data yesterday but not readiness?). Nonetheless, I am happy to take ideas for a design and pull request if anyone is willing to implement it.
As for HRV, I am not sure how to obtain it. Readines contains a score for it (score_hrv_balance), but I do not think this is the metric to which you are refering. Sleep data contains hr_5min, as well as hr_lowest and hr_average. I am assuming that they calculate it using hr_5min to calculate it, but I am not too sure how.

leikoilja commented 2 years ago

Sounds great, @nitobuendia 💥

How about creating a new sensor such as sensor.readiness in parallel with sensor.sleep which will extend the logic you already have in place for sleep? Just "one more" collective sensor with a bunch of data inside of it?

Edit: what's more perhaps be can just do even 3 sensors:

sensor.<unique_name>_sleep with the dump of sleep API endpoint data
sensor.<unique_name>_readiness with the dump of readiness API endpoint data
sensor.<unique_name>_activity with the dump of activity API endpoint data And what if unique_name comes from yaml config, so one can add more then one Oura ring per configuration. Let's say me and my wife has a ring each, so both data could be collected? :)

Right, HRV seems to be the same as score_hrv_balance in readiness API. :)

nitobuendia commented 2 years ago

I like the idea. Adding a bit more details on what this may entail:

There will be 3 new sensor files, one for OuraSleepSensor, one for OuraReadinessSensor and one for OuraActivitySensor. The current OuraSleepSensor should be moved from sensor.py to its own file.
Potentially, there would be an additional file with a base OuraSensor from which this would inherit the common configuration and methods (__init__, create_ouath_view, _get_date_type_by_name, _get_date_by_name, _get_backfill_date, async_update) and perhaps some of the properties (like name, state, and device_strate_attributes).
The new sensors would basically have its own update method and parsing. The rest would be inherited.
The endpoints already seem to be on API, but it would require new methods to make the calls to the endpoints.
There has to be a solution to avoid create_ouath_view to be called 3 times. Perhaps a singleton class handling this OAuth and shared across the 3 sensors could be a way to achieve it.
There might be some changes to the api.py file to make sure the _TOKEN_FILE has the same name on the 3 sensors.
Ideally, there would be some manifest configuration that would allow you to select which of the 3 data you want (sleep, readiness and/or acitivity). By default, all 3 are created.
The sensor.py would import all the 3 sensors (except the base) and set them up on async_setup_platform depending on whether they're on the configuration or not.
This would be a breaking change since the existing sensor would be named the existing name + _sleep, version should be increased to version 3.

leikoilja commented 2 years ago

Sounds like a great plan, @nitobuendia. Hope someone will have a time and desire to pick it up 💥

A small suggestion on point 7: I know you like all the .yml file configs and stuff, but once/if this component finally moves to a new way HA is handling integration (via config flows etc), for the integration configuration it would be great to use Options from Integration UI panel instead of manifest file :)

nitobuendia commented 2 years ago

Feel free to add a separate feature request for the config flow, for documentation purposes. Nonetheless, I do have strong feelings on the topic :)

chrismazanec commented 1 year ago

hello, I would love to have this - in my opinion, one of the most important oura metrics - implemented. is there any plan to implement it? thank you

akeslo commented 1 year ago

@chrismazanec I too wanted this so I took a pass: https://github.com/akeslo/ouraring-ha. Working well on my end but absolutely a duct tape job that needs to be refined.

nitobuendia commented 1 year ago

I have started the work to bump the API version to v2 (#15) and also do the changes mentioned in comment 4. Once these are ready, I will probably be adding all the endpoints available on the API documentation.

/cc @chrismazanec @leikoilja

nitobuendia commented 1 year ago

This is now in progress and published to branch oura-v2. At the moment, I've only added the readiness, but I will be doing further commits over this and maybe next week. The objective is to bring all the Oura API data as sensors.

The new component requires a different configuration, and has breaking changes. So if you want to use it, you may want to completely reinstall it from scratch until it's ready. The code is still experimental and subject to change. It will be merged after adding all other sensors and data points. Until then, assume it can change at any time.

@akeslo I know you've done good work on your integration and thought through data structures. If you feel like anything of it is worth merging here, happy to take ideas to integrate. I do not use most other data other than sleep, so I am sure the code can use your help if you're willing to contribute to it.

nitobuendia commented 1 year ago

@leikoilja @chrismazanec @akeslo Both the activity sensor and the readiness sensors have been implemented on the oura-v2 branch. If you could try testing the code and see if it does what you needed, check how easy it is to install, and report any issues - I would highly appreciate it.

I have tested it and it seems to work for me. However, do note that I am still adding commits to that branch, so it could be broken at times (if I push a wrong commit).

I am not marking this as fixed yet until I merge the code into the main branch.

bridgewatermedia commented 1 year ago

I'm not sure if a battery level variable is exposed to the API, but if it is, would it be possible to add that as well at some point?

nitobuendia commented 1 year ago

@bridgewatermedia Thanks for sharing your feedback. Since the request is not related to Readiness / Activity, could I kindly ask you to open a new issue? I am happy to have a look and check if it's on the API after that.

bridgewatermedia commented 1 year ago

Sorry! No problem. New to GitHub and the development world with all of it's intricacies/rules. My apologies. I appreciate the gentle rebuke. :)

akeslo commented 1 year ago

@leikoilja @chrismazanec @akeslo Both the activity sensor and the readiness sensors have been implemented on the oura-v2 branch. If you could try testing the code and see if it does what you needed, check how easy it is to install, and report any issues - I would highly appreciate it.

I have tested it and it seems to work for me. However, do note that I am still adding commits to that branch, so it could be broken at times (if I push a wrong commit).

I am not marking this as fixed yet until I merge the code into the main branch.

@nitobuendia Awesome work! Install was simple...I'm digging into the data now.

akeslo commented 1 year ago

@nitobuendia I may be misunderstanding, apologies if so, how would one get today's data?

My ring has data for today, 01/08/2023:

Yesterday points at 01/07/2023, as I did not wear it, the below makes sense:

How would I poll for Today, 01/08/2023? If I put "Today" or "0d_ago" the sensor returns null for those days.

Thanks.

nitobuendia commented 1 year ago

Thanks, @akeslo - that's odd.

When it comes to the Sleep sensor (not the Sleep Score), today data on the app usually equals yesterday data on the API.

Nonetheless, I think 0d_ago would match today if you want to test that. Give it a try if you're open or have a test environment. If that works, I am happy to see if I can easily add "today" in the future to make it easier.

A few other ideas to try if the above does not work:

The API does not read the App data but the Cloud data. If you go to https://cloud.ouraring.com - do you see the data there? If not, then the issue is that the app is not syncing to the cloud. I had this before and it was very annoying - although things have improved a lot in the last couple of years.
The sensor screenshot you shared seems to be for the activities sensor, however the Oura app screenshot seems to be for the sleep sensor. Have you checked the sleep sensor instead? If it helps, could you share your full configuration of Oura (without the token)?
Related to (2), there was a bug on the code where all the sensors were being created if not configured. Perhaps that's what is happening here: you may be checking the activities sensor which was created without you explicitly adding it to the configuration due to the bug, instead of looking at the sleep sensor which would contain the data that you've added on the screenshot. Updating the custom component code should be enough to fix this. No need to change the configuration, although maybe you need to delete the auto-created (now old) sensors.

akeslo commented 1 year ago

Thanks @nitobuendia let me try some of this today. I do recall this oddity with the fork I was working on, I have to look back at what I did there but I think I was forcing "Today" to be "Yesterday" because it was the only way it made sense. I'll check that.

I do know about cloud.ouraring.com - the data was mirrored there so it wasn't a sync issue. Though I know exactly what you mean and I've seen this as well before.

I'll report back once I have some time today to take another look.

Thanks!

akeslo commented 1 year ago

@nitobuendia Ok taking a deeper look, this is what I see:

My Config

  - platform: oura
    access_token: XXXXREMOVEDXXXXX
    scan_interval: 7200
    sensors:
     sleep:
       name: oura_sleep_metrics
       max_backfill: 0
       monitored_dates:
        - 0d_ago
        - 1d_ago
        - 7d_ago 
     sleep_periods:
       name: oura_sleep_periods
       max_backfill: 0
       monitored_dates:
        - 0d_ago
        - 1d_ago
        - 7d_ago       
     sleep_score:
       name: oura_sleep_score
       max_backfill: 0 
       monitored_dates:
        - 0d_ago
        - 1d_ago
        - 7d_ago

sleep (oura_sleep_metrics)

0d_ago is returning null for today however I have data via cloud.ouraring.com for 2023-01-11
Spot checking a few metrics, the 1d and 7d are correctly mapping back to cloud.ouraring.com for the correct dates. For example, under 1d ago lowest_heart_rate: 56 this matches what I see on the portal for 2023-01-10

sleep_periods (oura_sleep)

0d is not an attribute despite it being in the config.
1d has repeated attributes
Data for 1d and 7d appear to be matching up to oura portal for 2023-01-10

sleep_score (oura_sleep_score)

This is working without any issues and is supporting 0d_ago "Today" "Yesterday"

nitobuendia commented 1 year ago

@akeslo Super detailed and super helpful. Thank you so much!

Summary: from all the comments, it seems there are two issues to track:

An issue with the sleep sensor and today's vs yesterday's data. More testing is needed to confirm what's going on. #26
An issue with sleep period where 0d is not being added - presumably because there's no data, but that's not the expected behaviour. #27

Sleep

0d_ago is returning null for today however I have data via cloud.ouraring.com for 2023-01-11

The way I understood Oura works is that if you check on Monday, you'll have Sunday (yesterday) data which includes the sleep from Sunday to Monday. The UI calls it today, but technically is yesterday's data. The sleep and yesterday's data is used for sleep scoring, which is available today - which is why you have data on 0d on sleep_score.

However, I do see your data does not match and I can see the same on mine. I will have a look.

Spot checking a few metrics, the 1d and 7d are correctly mapping back to cloud.ouraring.com for the correct dates. For example, under 1d ago lowest_heart_rate: 56 this matches what I see on the portal for 2023-01-10

Awesome! :)

Sleep Period

0d is not an attribute despite it being in the config. This sounds like a bug. We usually fill in with an empty sensor when no data is registered. This type of sensors where you can have more than one value per day is new, so the logic to backfill that may have gaps. I will check.

1d has repeated attributes They are not repeated, the data is different. You can see that there's a - before average_breath indicating that's a new element within an array.

This sensor is called periods it contains all the sleeping periods detected by Oura. If you see the sample on the documentation, you can see that the data also has more than one sleep period.

Data for 1d and 7d appear to be matching up to oura portal for 2023-01-10

Awesome! :)

Sleep Score

This is working without any issues and is supporting 0d_ago

Awesome!

nitobuendia / oura-custom-component