Trying to activate media player on Rest+ Gen2 shows Failed to Call Service

spike-rogers commented 4 months ago

Trying to activate the play button for media is showing "Failed to call service media_player/media_play. list index out of range.

I'm trying to activate any sound from the device.

jdisom013 commented 3 months ago

Agreed, I'm having the same issue as well. See log past below. Logger: homeassistant.helpers.entity Source: helpers/entity.py:1520 First occurred: January 10, 2024 at 12:28:12 PM (14 occurrences) Last logged: 9:22:14 AM

Entity None (<class 'custom_components.ha_hatch.riot_media_entity.RiotMediaEntity'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <MediaPlayerEntityFeature.VOLUME_SET|VOLUME_STEP|STOP|PLAY|SELECT_SOUND_MODE: 87044>, please create a bug report at https://github.com/dahlb/ha_hatch/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation Entity None (<class 'custom_components.ha_hatch.rest_media_entity.RestMediaEntity'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <MediaPlayerEntityFeature.PAUSE|VOLUME_SET|VOLUME_STEP|STOP|PLAY|SELECT_SOUND_MODE: 87045>, please create a bug report at https://github.com/dahlb/ha_hatch/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation Entity None (<class 'custom_components.hacs.update.HacsRepositoryUpdateEntity'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <UpdateEntityFeature.INSTALL|PROGRESS|RELEASE_NOTES: 21>, please create a bug report at https://github.com/hacs/integration/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation

jdisom013 commented 3 months ago

Agreed, I'm having the same issue as well. See log past below. Logger: homeassistant.helpers.entity Source: helpers/entity.py:1520 First occurred: January 10, 2024 at 12:28:12 PM (14 occurrences) Last logged: 9:22:14 AM

Entity None (<class 'custom_components.ha_hatch.riot_media_entity.RiotMediaEntity'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <MediaPlayerEntityFeature.VOLUME_SET|VOLUME_STEP|STOP|PLAY|SELECT_SOUND_MODE: 87044>, please create a bug report at https://github.com/dahlb/ha_hatch/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation Entity None (<class 'custom_components.ha_hatch.rest_media_entity.RestMediaEntity'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <MediaPlayerEntityFeature.PAUSE|VOLUME_SET|VOLUME_STEP|STOP|PLAY|SELECT_SOUND_MODE: 87045>, please create a bug report at https://github.com/dahlb/ha_hatch/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation Entity None (<class 'custom_components.hacs.update.HacsRepositoryUpdateEntity'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <UpdateEntityFeature.INSTALL|PROGRESS|RELEASE_NOTES: 21>, please create a bug report at https://github.com/hacs/integration/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation

Agreed, Same issue for me with the Rest+.

lxqikrana commented 3 months ago

+1 Same issue on the Rest+ 2nd Gen. I can control the sound volume and light, but no audio control at all. There used to be an option to select sound mode, but that seems to have disappeared recently.

ddewar commented 3 months ago

Same issue here. This is the error I am getting when trying to make a service call:

`service: media_player.select_sound_mode target: entity_id:

media_player.my_rest_media_player data: sound_mode: "Ocean" `

Stopped because an error was encountered at February 3, 2024 at 2:57:47 PM (runtime: 0.04 seconds)

list index out of range

joemcmonagle commented 2 months ago

Having same problem here. Definitely big impact on our sleep time and wake up routines. Does anyone know of it works to rollback to an earlier version?

cmerkle commented 2 months ago

I noticed that there isn't even a sound mode attribute of the rest+ gen 2 media player entity anymore

volume_level: 0.95 media_content_type: music device_class: speaker friendly_name: My Rest+ Media Player supported_features: 87044

How am I supposed to set the sound mode or favorite preset?

zichaj commented 2 months ago

Just purchased the HatchRest 2nd Gen. I too get the "Error running action. Entity media_player.my_rest_media_player does not support this service" when trying to turn on sound from an automation. When trying directly from the Hatch Rest Mini / Plus integration I get "Failed to call service media_player/media_play. list index out of range.

Cre8ivejake commented 2 months ago

Yep, same issue here. Any word on if this is getting resolved soon? It’s broken quite a lot of our routines. I’m guessing it has something to do with the new software update that hatch rolled out on their end?

YayC commented 2 months ago

I updated our hatch rest+ from the mobile app yesterday and it broke Media player: Play, trace is:

  File "/usr/src/homeassistant/homeassistant/components/media_player/__init__.py", line 820, in async_media_play
    await self.hass.async_add_executor_job(self.media_play)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ha_hatch/riot_media_entity.py", line 55, in media_play
    self.rest_device.set_favorite(self._attr_sound_mode_list[0])
                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

I also only see these attributes in developer tools > states:

dahlb commented 2 months ago

@w1ll1am23 added the rest+ gen2 feature, I don't have a gen2 so I don't work on that feature.

If another developer would like to take up this issue, I am happy to answer any developer questions about the current codebase, and https://github.com/dahlb/hatch_rest_api is where the fix will likely start.

ChrisCarini commented 1 month ago

@dahlb - do you have any tips/tricks/guidance on debugging / testing the necessary changes? I've started to dive into this one, and made some progress, but I'm open to any guidance you have to share here! Please let me know!

dahlb commented 1 month ago

when I'm working through api changes I like to tweak https://github.com/dahlb/hatch_rest_api/blob/main/src/hatch_rest_api/stub.py for quickly checking if my changes work

also sometimes api changes can be ported from https://github.com/dgreif/homebridge-hatch-baby-rest

I also checkout this repo into ~/Desktop and use this docker to quickly test ha repo changes

version: '3.8'

services:
  homeassistant:
    container_name: homeassistant
    image: "ghcr.io/home-assistant/home-assistant:stable"
    volumes:
      - /Users/user/Desktop/ha_config:/config
    restart: unless-stopped
    ports:
      - 8123:8123

dacmcbibs commented 1 month ago

@dahlb - I have a rooted test device with the app on it and can capture the API calls to the Rest+ Gen 2, any interest in partnering on a fix?

kylebjordahl commented 1 month ago

I have a rooted test device with the app on it and can capture the API calls to the Rest+ Gen 2, any interest in partnering on a fix?

I'm interested in helping as well, if you can share any captured api calls I'm glad to write a PR against either this repo or the core api repo as necessary.

@ChrisCarini Did you manage to make any progress?

(Caveat: I have a 3 month old on my hands here so my availability comes in strange bursts)

ChrisCarini commented 1 month ago

If API calls can be shared (personal information removed, please), that'd be great! See below.

I actually have fixed it (at least, the features I use with the media player) as of a few weeks ago, and was starting to clean up the code for PR, but as I'm sure anyone with kids can imagine, it's very difficult to test the changes when having a little one napping (and thus, the hatch is in use). From memory, there is a small change in both the lib and this repo that need to be made.

I'd love to have others' API calls if possible, as it would help me build confidence that the change I have works for others, and not just myself.

@dahlb - I also started cleaning up the code, trying to simplify/dedup where I could, as well as adding more detailed debug logging (it helped me a ton when trying to figure out the necessary changes to be made). Would you prefer I get in the fix PR first so that it can be released and tested by others, and then submit PRs for the other changes, or what would your preference be? Anything's fine with me, just let me know your pref.

vans113 commented 1 month ago

Just got the rest + gen 2 also. I can help beta test anything anybody has.

dacmcbibs commented 1 month ago

@ChrisCarini - if you can give me a sense of which calls you're interested in (this thing seems to make tons) I would be more than happy to share what I am seeing to validate what you've been working on , are you looking for the GetSoundContentByHatchID call? (that looks to be the one that makes the most sense to me)

ChrisCarini commented 1 month ago

@dacmcbibs - heres some educated guesses based on what I'm seeing:

May sound silly, but what's the hostname that's being used for the calls?
- I ask, because the python lib code uses https://data.hatchbaby.com/ throughout, but I saw the homebridge/ts code the python lib is based on switched to use https://prod-sleep.hatchbaby.com/ ~1 mo ago. Both resolve to the same two reals, so it should be identical, as far as I can tell. My calls in my test version are using data., but if prod-sleep. is what the app is using, I'd like to propose we switch the python lib over to that as well.
(HTTP GET) Both the request (reminder, obfuscate any auth tokens or other identifying information, please!) and the response of any calls to service/app/routine/v2/fetch
- based on what I recall seeing, the request used to specify the type to be favorites, but that is no longer valid (so the response is empty with the current lib code)
- I also noticed that In the updated app, I didn't see any 'favorites' anymore, so I'm thinking they killed this and may have replaced it with something else.
- ⚠️ NOTE: If anyone owns a Restore device, if you could capture the same for me, that'd be great - the code alludes to the type being 'routine', but I would like to confirm.
MQTT traffic to AWS (possibly to https://cognito-identity.<REGION>.amazonaws.com), anything related to 'get shadow accepted'
- sorry, but my familiarity of MQTT is not nearly as strong/deep as other protocols 😓 ).
If there are any requests you see that show a listing of audio track information, that'd be great!
- sorry, I don't have much helpful here, I had to 'trial-and-error' every audio track from the app
- I found 19, but wondering if there might be more; one example to maybe search for is "Brown Noise"; maybe search these strings and see if you happen to find a call that lists all of them?
  - 10200
  - Brown_Noise_2_20210412.wav

Sorry if that's a bit long - thats what comes to my mind after looking through my local code I have thus far. If I think of anything else, I will let you know.

dbolton commented 1 month ago

Happy to capture api call data if it isn't too complicated. I would need guidance on how to do it though.

dacmcbibs commented 1 month ago

If there are any requests you see that show a listing of audio track information, that'd be great!

sorry, I don't have much helpful here, I had to 'trial-and-error' every audio track from the app

I found 19, but wondering if there might be more; one example to maybe search for is "Brown Noise"; maybe search these strings and see if you happen to find a call that lists all of them?

10200

Brown_Noise_2_20210412.wav

Sorry if that's a bit long - thats what comes to my mind after looking through my local code I have thus far. If I think of anything else, I will let you know.

Sorry for the delay... my little one has been sick

I'll start with your third item. A call to this endpoint seems to return a list of potential sounds https://graphql.contentful.com/content/v1/spaces/hlsdh3zwyrtx

it does this call when navigating to the "library tab" in the app

the JSON contains 10 errors in it (which is interesting) but also contains objects for "colorCollection", "trackCollection", "channelCollection", "soundCollection", "subcategoryCollcetion"

my colorCollection, trackCollection, and channelCollection came back blank

for subCategory collections it returned the following -Tiny Bed Concerts -Music -Time-to-Rise -Time-for-Bed -Sleep Stories -Recently Added -Seasonal Stories -Dreamscapes -Guided Rest -Bedtime Roundup

each of these with the following -__typename
-sys
-name
-description
-beta
-tracksCollection
-icon
-channelsCollection

(the category names are the "names" entry

The soundCollection is the really interesting one, it includes the following fields: -__typename
-sys
-title
-description
-hatchId
-hidden
-theme
-filtersCollection -image
-portraitImage

returning the following list: title hatchId

Brown Noise 10200 Crickets 10148 Thunderstorm 10146 Forest Lake 10082 Calm Sea 10056 Campfire Lake 10195 Water 10142 Fan 10145 Ocean 10138 Vacuum 10198 Twinkle Twinkle Lullaby 10193 Wind 10141 Dryer 10143 Brahms Lullaby 10192 White Noise 10137 Rock-A-Bye Lullaby 10194 Heartbeat 10144 Rain 10139 Birds 10140

This call does not produce a list of wav files. Interesting point to note is that the query to the api endpoint includes the hash id , which is in the sys object of the soundCollection json as a variable, it also includes some subcategory id's in the query as well , the query string for tracks, channels, and colors are blank, which is likely why the json returned is blank.

This seems to imply that the app knows what sounds to look to look for, which may mean that I missed a call that identifies the sound id's available. I suppose its possible they are hard coded in the app (that's not how I would design it ;) ) I'll see if i can get a fresh capture.

@ChrisCarini - do you want to work on this on a discord server, or have some other communication preference ?

dacmcbibs commented 1 month ago

@dacmcbibs - heres some educated guesses based on what I'm seeing:

May sound silly, but what's the hostname that's being used for the calls?

I ask, because the python lib code uses https://data.hatchbaby.com/ throughout, but I saw the homebridge/ts code the python lib is based on switched to use https://prod-sleep.hatchbaby.com/ ~1 mo ago. Both resolve to the same two reals, so it should be identical, as far as I can tell. My calls in my test version are using data., but if prod-sleep. is what the app is using, I'd like to propose we switch the python lib over to that as well.

@ChrisCarini - I haven't seen any calls to https://data.hatchbaby.com/ - but I have seen calls to https://data.emb-api.com

I do see calls to https://prod-sleep.hatchbaby.com/ - examples of what I am seeing

get list of devices,
a "devicePhase" request that ends in the MAC address (whatever that does)
a provisionedinfo request again against the MAC address
Token request
subscription check
a riotPlus endpoint that again uses MAC address to pull info about the device (e.g clock settings, current color, current sound, etc.)
a call that looks to return routines
firmware version lookup (including download link)
looks like some logging requests
app version checks

Also, if I navigate around the app options enough I can get it to give me wav files that go to each sound (looks like as you navigate into the sound it looks up the path)

I'll have to do a few more captures when my little one is not sleeping , but I did manage to grab one mqtt packet today: to wss://aacsjje0gbbcc-ats.iot.us-west-2.amazonaws.com

kylebjordahl commented 1 month ago

Ok, I think I have a path to a solution which starts in the API library, but need someone to test against other devices to see if my change works there as well. I have tested with my Rest Gen 2, but if you have a different model and are experiencing this issue, please check out this PR and run the stub.py script. Note that the script will cause sound playback at low levels

dacmcbibs commented 1 month ago

We have a Rest+ Gen 2. ~~so I expect it to behave identical to your Rest Gen 2, but I will run through your PR in the morning to see if I can verify your results~~. Confirmed! this looped through sounds on our device. @kylebjordahl Thanks for putting this together!

ChrisCarini commented 1 month ago

@ChrisCarini - do you want to work on this on a discord server, or have some other communication preference ?

@dacmcbibs - I'm open to it! I don't use many/any that often, but I'm happy to hop on one to iterate on this faster. @kylebjordahl - since you look like you're working on this too, are you open to this as well?

@kylebjordahl - what do you get for the payload response for 'sounds'? One thing I'm not content with right now is that this API is not returning all the tracks; specifically, my response from the API is missing the following:

BrownNoise = 10200    # https://assets.ctfassets.net/hlsdh3zwyrtx/Bqk8q7mjFcSa8B1Ovgllp/e9701ae7df057a31b89a4cd2830ef0dc/Brown_Noise_2_20210412.wav
Vacuum = 10198        # https://assets.ctfassets.net/hlsdh3zwyrtx/5mg3e3BtpIn0YaQfOVVNRJ/528e94cfd7232481637fd3ce7c7141f2/Industrial_Vacuum_Cleaner_20191220.wav
ForestLake = 10082    # https://downloads.ctfassets.net/hlsdh3zwyrtx/2WgzZNttwX5RK4twPtMCsS/64de4333300711282b42046020fc3aa0/Forest_Lake_20191220.wav
CalmSea = 10056       # https://assets.ctfassets.net/hlsdh3zwyrtx/1LelwPIVm5YZle7WP42u2X/b26f1d8a35b4c083a0bb65c9e323b7a7/Calm_Sea_20191220.wav
Crickets = 10148      # https://assets.ctfassets.net/hlsdh3zwyrtx/5X1S7xtEHyZab67wRbsEda/92f8bc6c927a384bd2262ebc6999465a/010_crickets16.wav
CampfireLake = 10195  # https://assets.ctfassets.net/hlsdh3zwyrtx/6Gb9MNlL9VcMcUmo4jzCSv/c457b63210359467e729fe7c1d624edd/Campfire_Lake_2_20210412.wav
Brahms = 10192        # https://assets.ctfassets.net/hlsdh3zwyrtx/2XXRwK0Xqw1KLBr28RIkSe/ee6af976c9980823389134eeded7f07b/011_brahms16.wav
Twinkle = 10193       # https://assets.ctfassets.net/hlsdh3zwyrtx/69qMR6Wp2hPD7gk7hSfRl5/25af4cefe997d5ba4070e71ae21e7eb3/013_twinkle16.wav
RockABye = 10194      # https://assets.ctfassets.net/hlsdh3zwyrtx/7lY2LJerpBhO7vravoQ14J/debcf202883c61eaa384ee826dec4026/014_rockabye16.wav

Note: These additional tracks show up in the app, and if I browse to them / play them from the app, I can grab these wav URLs.

If I hard-code a map for these and call _update() w/ the respective payload to set an audio track (passing ID & URL of the wav file), these tracks above work fine. Also, FWIW, I do not have any paid membership for Hatch - all the audio sources I use from the app are free (so I do not think these sounds are limited to paid accounts).

Aside from that, the changes I have that I've been using are very similar to the PR you posted, however, the PR you posted seems to be missing a few bits to get the 'favorites' portion to continue to work. Have you been able to test / work-on this portion yet?

ChrisCarini commented 1 month ago

Oh, one other question! :)

@dacmcbibs / @kylebjordahl - right now, I have the modified custom component and within it's python module, the modified lib (so, they both are within the same directory tree, not in the same directory). This is for ease so I can just SCP the whole directory over to HA and restart HA (script below).

#!/bin/bash

HA_SSH=<user>@<hostname>

scp -r \
  -i ~/.ssh/<MY_SSH_KEY> \
  -P 22 \
  ./custom_components/ha_hatch_MY_MOD/* \
  $HA_SSH:/root/homeassistant/custom_components/ha_hatch_MY_MOD/

ssh $HA_SSH "source /etc/profile.d/homeassistant.sh && ha core restart"

This makes development easier, but not amazing. My current gripes with this are (1) I have to reboot HA each time I make a change I want to test (this might be unavoidable, no?), and (2) I've had to modify the custom components imports to reference the 'local lib files' instead. This is making it harder to just have branches of each that I can share to collaborate on.

❓ Any tips on a nicer development workflow where I don't have to have modified the imports to a local copy of the lib?

ChrisCarini commented 1 month ago

@dahlb - Below are two PRs that I believe should address this issue

PRs

https://github.com/dahlb/hatch_rest_api/pull/22 - the lib changes
- I see https://github.com/dahlb/hatch_rest_api/pull/21 was merged in already :p - I've merged the changes from that into my branch (which still has several applicable changes that support the PR below in (2)).
https://github.com/dahlb/ha_hatch/pull/121 - the custom component changes

Note: These both work towards supporting specific sounds + user favorites (as configured in-app).

Testing Notes

There was only one minor nuance in my testing, but everything else I could think to test worked fine 🎉 ; details below:

✅ Attributes for the media player now show as below: Note: Notice the sound_mode_list shows the default sounds + the favorites. I thought this was nice for users of HA as they can see the available options to set for sound mode.
✅ 'Pause' & 'Play' both work as expected; going from pause -> play will resume the audio that was previously playing
✅ Selecting sound mode works for default sounds (e.g. Rain or Water), as well as custom favorites (the last 3 in the above screenshot)
🤔 In testing, sometimes going from "playing favorite 'sound mode' option from an automation/script" to "default sound (e.g. 'Rain') option from the developer tools services tab" would not work without pausing in between (this happened only a few times, and was inconsistent, which was making me think it was more likely an issue w/ the device network connection (e.g. wifi) and/or me sending too many commands too quickly. I only want to mention it in case). In the case of one running into this issue from an automation, the easy work-around would be to pause before setting the sound (or, some combination of pause/play/set sound mode).

Questions for the audience/maintainer

If there is anything else people want me to test out (as far as automations/services/workflows), please let me know. My general use-case is fairly simple - I just have an automation that starts playing a specific favorite I've configured in-app as part of one piece of the HA automation (also dim lamp in room, etc).

If there are some more specific / nuanced flows people have and want me to test via an automation / service call, please let me know (and, be as specific as you can, including any custom settings that may need to be set in-app).

lxqikrana commented 1 month ago

I'm going to ask a stupid question here - how do I take @ChrisCarini 's changes for a spin in my HA setup? I see there are two different PRs involved. How do I merge that in my custom_components/ha_hatch dir to get this running?

Thanks!

ChrisCarini commented 1 month ago

I'm going to ask a stupid question here - how do I take @ChrisCarini 's changes for a spin in my HA setup? I see there are two different PRs involved. How do I merge that in my custom_components/ha_hatch dir to get this running?

Thanks!

Not a stupid question - I actually have the exact same question!

Also, don't take ^ and infer that to think "Wait, does that mean @ChrisCarini didn't actually test any of this in the real-world??" In fact, I did, but the way I did it was pretty hacky (at least, I think) - the TL;DR: is I:

copied the library code (since it's only a handful of files) into a new directory in this project
updated all the imports in this project to point to the copied-lib
updated various metadata (like metadata.json) to have a domain of ha_hatch_MY_MOD
scp'd everything over to my HA instance in the custom_components dir (I scripted this, because of how frequently I was making tweaks to try different things 😅 )
restart HA
add/refresh the integration w/ this 'new' custom component (this yields new everything for your Hatch, likely with _2 added to the end)
update/copy a few of my automations that use Hatch to reference the new entities (i.e. _2)
start testing things

Note: I really only had to do 4 & 5 (and sometimes 6) over and over as I was making changes/tweaks. Not too bad, but not super-smooth, either.

If there's a better way, I'd love to know! :) Please teach me! 🙏

dacmcbibs commented 1 month ago

@lxqikrana - Also make sure that you update your const.py file so that your DOMAIN matches your folder name (whatever you called it) . I also had a heck of a time getting python to recognize the module in a sub folder (must have a pathing issue somewhere in my environment) , so I moved it into my site-packages - which allowed me to load and sign into the integration. I have it all set up to test tomorrow.. My little one is sleeping at the moment.

@ChrisCarini - unfortunately I think until your rest_api changes are merged and available for download from the repository the way you're doing this is probably as good as it gets, the other option would be to overwrite the rest_api files in your site-packages folder so you do not have to modify the code to reference your module. I haven't found a better way

ChrisCarini commented 1 month ago

@ChrisCarini - unfortunately I think until your rest_api changes are merged and available for download from the repository the way you're doing this is probably as good as it gets, the other option would be to overwrite the rest_api files in your site-packages folder so you do not have to modify the code to reference your module. I haven't found a better way

Gotcha - this might sound silly, but where does site-packages get placed on the file system? I don't recall where exactly I looked, but I couldn't find it and just opted to do the way I mentioned in pursuit of debugging/modifying faster. 😅

dacmcbibs commented 1 month ago

I run a dockerized version of Home Assistant, in the docker container the site-packages are located here: /usr/local/lib/python3.12/site-packages/

The downside to this approach is that if you restart the docker container the changes in site-packages will be lost. Restarting the Home Assistant app by navigating to developer tools they will survive. (which is how I got them to load) so you may have to copy them back again on a full reboot, which in theory is just a modification to your shell script. You could also put the folder in your config folder and then write a script that runs on startup to copy the files to the folder, either way a reboot would still likely be needed, but at least its less code modification to test

dacmcbibs commented 1 month ago

@ChrisCarini - Great work! I just gave this a quick run through and had zero issues on our rest+! It actually works a bit better than the current integration - with the current integration if I wanted to turn the light on, and play sound and set the volume I had to pause between, and even then sometimes I would need to set the volume twice. So far with your code I can run the steps back to back and it hasn't failed. I haven't had any issues flipping between sounds that you saw in your testing. I also made sure to test the clock, light, and toddler lock controls - no regressions there. All sensors working as expected

zichaj commented 1 month ago

Great work guys! I have a HatchRest 2nd gen from Amazon. My return window closes today so I've got to get it returned if there is no line of site to a fix. Is this now at a point where you are confident this work will get merged back into the main branch and a new custom integration version update released? If it's just a matter of time I'll keep this otherwise I'll return it and purchase it again later once resolved.

Also, I can test on my setup too if that would be helpful.

Thanks!

On Sat, Mar 30, 2024 at 8:13 AM dacmcbibs @.***> wrote:

@ChrisCarini https://github.com/ChrisCarini - Great work! I just gave this a quick run through and had zero issues on our rest+! It actually works a bit better than the current integration - with the current integration if I wanted to turn the light on, and play sound and set the volume I had to pause between, and even then sometimes I would need to set the volume twice. So far with your code I can run the steps back to back and it hasn't failed. I haven't had any issues flipping between sounds that you saw in your testing. I also made sure to test the clock, light, and toddler lock controls - no regressions there. All sensors working as expected

— Reply to this email directly, view it on GitHub https://github.com/dahlb/ha_hatch/issues/95#issuecomment-2028031088, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5RLGANOVOMWYIOOEIPJKPTY22T4ZAVCNFSM6AAAAABBBI6VFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRYGAZTCMBYHA . You are receiving this because you commented.Message ID: @.***>

ChrisCarini commented 1 month ago

I run a dockerized version of Home Assistant, in the docker container the site-packages are located here: /usr/local/lib/python3.12/site-packages/

The downside to this approach is that if you restart the docker container the changes in site-packages will be lost. Restarting the Home Assistant app by navigating to developer tools they will survive. (which is how I got them to load) so you may have to copy them back again on a full reboot, which in theory is just a modification to your shell script. You could also put the folder in your config folder and then write a script that runs on startup to copy the files to the folder, either way a reboot would still likely be needed, but at least its less code modification to test

Awesome, this is super helpful! I'll give this a shot next time I'm able to 'borrow' the Hatch to keep working on this fix! 🙇

@ChrisCarini - Great work!

Thank you!! 🙏 🙇

I just gave this a quick run through and had zero issues on our rest+!

That's exciting news, and makes me feel more confident in the changes! I appreciate any/all real-world testing of these changes!

It actually works a bit better than the current integration - with the current integration if I wanted to turn the light on, and play sound and set the volume I had to pause between, and even then sometimes I would need to set the volume twice. So far with your code I can run the steps back to back and it hasn't failed.

Hmm, thats quite interesting that you say that - I actually have had to do the same (pause between) w/ the 'current' integration, but didn't think to test my scripts without that pause. I'll add that to my list of things to test - appreciate you testing it and calling it out!

I haven't had any issues flipping between sounds that you saw in your testing.

Ok, that's great and encouraging to hear!

I've started to wonder if something might be up w/ my HA install, as I've noticed the past few days that the scripts I have that interact w/ the Hatch seem to be reverting to the 'old' entities for the Hatch on the 'current' integration, and not using the 'new' entities for the Hatch w/ my 'modified' integration (aka, with these changes). Because of this, I've also noticed that sometimes my scripts have issues - it's been a bit frustrating because I keep thinking "ahh, maybe my change/code or hatch API changed", only to look into the scripts to find out that the entity is wrong. I'm not sure why, because I keep changing it and saving the scripts. No matter, glad to hear you're not having the sound issue I was seeing! Thank you for testing it!

I also made sure to test the clock, light, and toddler lock controls - no regressions there. All sensors working as expected

Perfect! Also glad to hear this as well! I personally have scripts that tweak the clock brightness, light color & brightness, and sound. All of those have been working without issue (with the exception of the weirdness I mentioned above about the entities in my scripts seemingly reverting to the 'old' ones - never had an issue w/ the HA controls when HA is pointing at the right entity). Earlier today, I realized that I should do more explicit testing of all the other features, so that's been added to my "todo" list for these changes, however, I'm glad you've had a chance to test these and haven't found any issues. Appreciate it!

ChrisCarini commented 1 month ago

Great work guys!

Thanks @zichaj for the kind words!

Is this now at a point where you are confident this work will get merged back into the main branch and a new custom integration version update released?

I can't speak to if this work will get merged in or not, that's up to @dahlb who maintains/owns the two repos. :)

I certainly hope these changes do get merged in / this issue is fixed - that's my ultimate goal/ideal here, after all. I'm pretty motivated to have my Hatch work with my HA setup (hence me spending the countless hours of time to learn this code, debug it and the Hatch payloads, and put up the PRs w/ the solutions I've come to 😴 ). That being said

I have a HatchRest 2nd gen from Amazon. My return window closes today so I've got to get it returned if there is no line of site to a fix. ... If it's just a matter of time I'll keep this otherwise I'll return it and purchase it again later once resolved.

Understandable. My personal advice would be to keep it - the Hatch has become a key part of my day-to-day given the young-one, and like I mentioned above, I'm personally very motivated to get it working with HA. If absolutely need be, I'd be willing to have a fork w/ the needed changes, although I'd much rather just contribute back to the main repo(s) (the path I'm currently pursuing). As of right now, I don't see any reason this issue wouldn't get fixed - there are several folks interested in seeing it fixed, active work being done, PRs up for review, others testing those changes, and the author/maintainer isn't 'inactive'.

Also, I can test on my setup too if that would be helpful. Thanks!

Would love to have more folks test - please feel free to, if you feel comfortable testing the changes! 🙏

lxqikrana commented 1 month ago

Thanks @ChrisCarini and @dacmcbibs for the detailed instructions. I think that is a bit beyond my skill level. I will just wait for @dahlb to merge in these changes. Hopefully that is soon!

yuejon commented 1 month ago

I hope this gets merged soon. It's been annoying having to manage the sounds through the hatch app.

droans commented 1 month ago

@ChrisCarini

If the manifest.json contains all the proper requirements, you should be able to publish a release for others to use,

Create a zip file named ha_hatch.zip containing the files in custom_components/ha_hatch. The structure should be:

ha_hatch.zip
  - __init__.py
  - binary_sensor.py
  ...

Then, publish the release with that zip file attached. Anyone else can add it to their HACS install and download it.

Home Assistant/HACS should check the requirements in the manifest.json file and ensure that the system has the correct dependencies.

ChrisCarini commented 4 weeks ago

@droans - cool, thanks, this is helpful!

Ok, https://github.com/dahlb/hatch_rest_api/pull/22 is now merged, so it's easier to share changes with folks for more broad testing.

Testing Instructions

Below assumes you:

have HACS already installed and setup; if not, please seek instructions on how to do so
are using the latest version of HA (2024.4.1)

Instructions

[!CAUTION] Following the below steps, you are about to download a ZIP file with arbitrary code that a stranger (to you) uploaded to the internet.

Proceed with caution!

Best-practice says you should first scan the code to ensure it does what you expect, and nothing malicious (it doesn't do anything malicious, but don't just trust me, double check the code for your own verification). The place to do this would be before step 6 below.

Open HACS in the left menu bar
Click on Integrations
Click the 3 dots in the upper right; click Custom repositories
Enter https://github.com/ChrisCarini/ha_hatch for Repository, and Integration for Category; then click Add
Search for 'ChrisCarini_MOD'; click the one option that shows; click Download in the bottom right; click Download in the modal that appears.
Restart HA
Navigate to Settings -> Devices & services
1. I'd recommend disabling the existing integration you have, just so there is no/less confusion when testing
Click Add Integration; search for ChrisCarini MOD
Enter your Hatch auth credentials; click Submit
New entities and devices should now be added; test accordingly

NOTE: The entity names may have changed, so you may need to update any scripts/automations you have to reference the new entities/devices accordingly.

Personal Testing Notes

I just upgraded to HA 2024.4.1 this evening, and tested as much as I could think of post-upgrade; table below:

Control	Manual	From Automation/Script
`Clock -> On/Off`	:heavy_check_mark:	:heavy_check_mark:
`Clock -> Brightness`	:heavy_check_mark:	:heavy_check_mark:
`Light -> On/Off`	:heavy_check_mark:	:heavy_check_mark:
`Light -> Brightness`	:heavy_check_mark:	:heavy_check_mark:
`Light -> Color`	:heavy_check_mark:	:heavy_check_mark:
`Media Player -> On/Off`	:heavy_check_mark:	:heavy_check_mark:
`Media Player -> Volume -> Slider`	:heavy_check_mark:	:heavy_check_mark:
`Media Player -> Volume -> Increase/Decrease`	:heavy_check_mark:	:heavy_check_mark:
`Media Player -> Play/Pause`	:heavy_check_mark:	:heavy_check_mark:
`Media Player -> Change Sound Mode`	:heavy_check_mark:	:heavy_check_mark:
`Toddler Lock -> On/Off`	:heavy_check_mark:	:heavy_check_mark:

If folks could please test and reply if they have different results, that'd be great!

Worth noting, you can use the media player to turn the device off via the power button, but clicking the power button to turn it back on doesn't do anything (no media plays). This is because right now there isn't anything implemented for the 'on' method. To get the media player to be responsive again, I was able to change a clock/light setting (increase brightness, for example) and then things worked. ❓ Question: Given the above, what are folks expected behavior for 'on'? I think this is something that could be added fairly easily. One idea I had was to turn on the first 'favorite'.

kylebjordahl commented 4 weeks ago

@ChrisCarini - To your question: I would expect that it would either a) Turn on the first "favorite" b) Restore whatever was last playing

I recognize that b is a larger lift, we would need to look at the state history and figure out what that was, so probably not worth the effort, but I can envision a world where my partner hits the off button accidentally and then hits the on button expecting to just restore state, and then gets very frustrated when something different starts playing.

That being said, I would be glad to try and implement b in a future PR if others think that functionality makes sense; it doesn't seem worth it to me to hold out on getting this fixed just for that UX case.

ps- I'll try and get a test done this weekend with our Rest Gen2 (in the scarce time it's not being used)

ChrisCarini commented 4 weeks ago

@kylebjordahl - thanks for your thoughts! (B) is actually already implemented in the play/pause buttons, so it's really not much more work. Maybe it's something to be configured? I asked my SO this am, and they also said (a). Will continue collecting folks thoughts - so please chime in!

vans113 commented 4 weeks ago

@ChrisCarini I'm trying to set this up to help test. I followed the above instructions but upon login it fails. I have triple checked username and PW and even signed out and back in again with the user and PW. I found this in the logs. Is it of any help? If not I can share more from the logs.

2024-04-06 15:29:10.156 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry ( deleted username ) for ha_hatch_ChrisCarini_MOD Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 551, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

ChrisCarini commented 4 weeks ago

@ChrisCarini I'm trying to set this up to help test. I followed the above instructions but upon login it fails. I have triple checked username and PW and even signed out and back in again with the user and PW. I found this in the logs. Is it of any help? If not I can share more from the logs.

2024-04-06 15:29:10.156 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry ( deleted username ) for ha_hatch_ChrisCarini_MOD Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 551, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@vans113 - are there more logs you can share? That snippet seems like it is cut off.

vans113 commented 4 weeks ago

@ChrisCarini See if this helps more.

2024-04-06 15:33:34.806 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry (Deleted ) for ha_hatch_ChrisCarini_MOD Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 551, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/ha_hatch_ChrisCarini_MOD/init.py", line 105, in async_setup_entry await setup_connection("initial setup") File "/config/custom_components/ha_hatch_ChrisCarini_MOD/init.py", line 77, in setupconnection , mqtt_connection, rest_devices, expiration_time = await get_rest_devices( ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/hatch_rest_api/util_bootstrap.py", line 31, in get_rest_devices token = await api.login(email=email, password=password) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/hatch_rest_api/hatch.py", line 66, in login await self._post_request_with_logging_and_errors_raised( File "/usr/local/lib/python3.12/site-packages/hatch_rest_api/hatch.py", line 17, in request_with_logging_wrapper response_json = await response.json() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/aiohttp_client.py", line 73, in json return await super().json(*args, loads=loads, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 1166, in json raise ContentTypeError( aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/plain; charset=utf-8', url=URL('https://data.hatchbaby.com/public/v1/login')

ChrisCarini commented 4 weeks ago

@vans113 - thank you, that is more helpful than the previous one you shared!

🤔 thats quite odd though - I'm reading that error to mean "we're trying to auth in config flow as a test, but the response from Hatch is not JSON" - and the changes here haven't touched the authentication flow at all.

I'll have to wait until tomorrow to be able to test entering an both a valid, and invalid username/password tomorrow to see the response for each.

I have triple checked username and PW and even signed out and back in again with the user and PW.

Just to clarify, do you mean you've signed out of the app (not HA) and back in again via the app? Or are you referring to the other integration or something else perhaps?

vans113 commented 4 weeks ago

@ChrisCarini

Just to clarify, do you mean you've signed out of the app (not HA) and back in again via the app? Or are you referring to the other integration or something else perhaps?

When it wouldn't work I thought maybe I entered the wrong user and PW. So I went to the hatch website and did a copy-paste to exactly what I was putting into HA and logged right in.

I'm happy to retest anything and send you my logs. Let me know how I can help and thanks for all the work on this!

ChrisCarini commented 4 weeks ago

@vans113 - hmm, that is quite odd. I'm assuming you get this if you try multiple times / if you try again now, right? I just tried authenticating again with debug logging turned on, and was able to auth fine for my account (both according to the UI, as well as the debug logs).

dacmcbibs commented 4 weeks ago

I reinstalled using HACS and also cannot reproduce the login issue either... @vans113 Let start with a few data points to help us troubleshoot:

Are you running HA OS, Container, Core or Supervised?
What version of Home Assistant?
Are you able to log into the unmodified version of the ha_hatch integration (the one that doesn't currently work with rest gen 2 devices) The auth flow did not change so we should expect a failure here too
Any special network configuration that may get in the way of the integration correctly talking to the Hatch API?
Are you comfortable navigating the command line?

Just trying to get a better sense of your config so we can eliminate potential break points.

vans113 commented 4 weeks ago

@dacmcbibs

Are you running HA OS, Container, Core or Supervised?

HA OS as a VM using ESXI.

What version of Home Assistant?

Core is 2024.4.1

Are you able to log into the unmodified version of the ha_hatch integration (the one that doesn't currently work with rest gen 2 devices) The auth flow did not change so we should expect a failure here too

Yes I am able to login with no issues on that one and use the light on feature all the time.

Any special network configuration that may get in the way of the integration correctly talking to the Hatch API?

No. I do have a fortinet firewall on the edge on my network but there is a Lan to Wan policy set to all for all services so nothing blocking that I see in logs also.

Are you comfortable navigating the command line?

Pretty comfortable yes

dacmcbibs commented 4 weeks ago

@vans113 I am really confused that the old one lets you log in and the new doesn't. If I understand correctly (I've never used it) - HA OS is basically a supervised docker instance - have you tried completely shutting down Home Assistant? In theory this should revert back any libraries added to the container (e.g. hatch_rest_api) and ensure that the latest gets pulled. Did you have the old one disabled when you tested? In theory both should be able to log in at the same time, but in my environment I saw once in a while saw odd behavior when I had both enabled if I refreshed the integrations

If you have access to the command line for Home assistant (ssh) you could try looking at the python site-packages to make sure you have the latest. In theory even if you don't you should still be able to sign in. Might not hurt (if you have ssh to the Home assistant container) to see if you can run stub,py to see if you can log in... (being that the old integration works I don't know what would happen here)

The one thing that does stand out is this line in your error:

aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/plain; charset=utf-8', url=URL('https://data.hatchbaby.com/public/v1/login')

I am pretty certain the mimetype should be returning as application/json. I just wrote a quick powershell script that does a login to get a token (python isn't my strongest language) , and the response is being returned as application/json

Might be interesting for you to (if you can get the response from the login) to look at the contents of the response to see what it returned... if you are going to share make sure you redact at least the following :

ID
email
babies (if populated)
FirstName
LastName
Sync
Token

and anything else that looks personally identifiable. The format of the response is the important thing here, the actual values are not required to understand what is being returned from the login API endpoint.

Just a few thoughts....

dahlb / ha_hatch