fondberg / spotcast

Home assistant custom component to start Spotify playback on an idle chromecast device as well as control spotify connect devices
Apache License 2.0
672 stars 97 forks source link

Music was stopped because spotify was used at a different device. #13

Open almostserious opened 5 years ago

almostserious commented 5 years ago

I have noticed that after some time when playing on Chromecast & Google Home Groups, that my playback stopped and the Google Home said: Playback stopped because spotify was used somewhere else. (It wasnt though). It didnt happend again after i commented out the sensor. Not sure yet why this could happen.

fondberg commented 5 years ago

I noticed this as well. If it is related to the sensor it would be weird but maybe I'm using the pychromecast wrong somehow. If someone can help debug this it would be great

almostserious commented 5 years ago

Actually today it also happened without the sensor..

fondberg commented 5 years ago

Hmmm weird. Was it after 60 mins? That is when the token expires. It should renew it internally in the chromecast device. If the sensor is not in the picture I guess the problem must lye outside the spotcast component

fondberg commented 5 years ago

@almostserious did you have the sensor enabled or not when this happens?

almostserious commented 5 years ago

So, yesterday it happend again. Sensor is NOT enabled. But it happend after roughly 60min.

fondberg commented 5 years ago

I got this when music was started with Google assistant voice command so I think it has something to do with the spotify app on the chromecast device when it fails to refresh the token

fondberg commented 5 years ago

Anyone got any ideas?

fondberg commented 5 years ago

I tested all kinds of events from pychromecast for the session but couldn't get anything for this scenario. That means that a fix for this would need to entail a couple of things in this code as well as in pychromecast. Here we need to implement some kind of timer to keep track of when the token becomes invalid and request a new token To send the new token we need to add functionality for it in pychronecast.

Phew.. it won't be simple... any volunteers to help out?

williamson10 commented 4 years ago

FWIW I have the same issue without even using your plugin. In my case it is that spotify on my phone hasn't refreshed in a while. If I try to control the cast from my phone too soon, I get this and my phone tells me it is playing locally. I usually know because the wrong song is displayed on my phone. If I wait for it to update, all is good.

I would guess something similar is happening here. Maybe even the same thing.

fondberg commented 4 years ago

I think I've seen this as well. I think it is due to the phone app doesn't update the token either. Solving it is not that easy though...

ezequiellop commented 4 years ago

Hi, I have the same problem. After 60min the playback stops. Is there any solution?

fondberg commented 4 years ago

Read the comments above

fondberg commented 4 years ago

It could be worth investigating the traffic between a phone and a cast device and see if the phone sends either a refresh token at start or sends anything after a certain amount of time

MagicMicky commented 4 years ago

@fondberg do you know if the traffic between device and chromecast is encrypted via tls? I can try to setup wireshark later this evening (eu timezone) to see if I can get anything

fondberg commented 4 years ago

It follow the dial protocol but I'm not sure if it is encrypted. A simple wireshark will probably show that.

fondberg commented 4 years ago

Or maybe I'm wrong. It seems chromecast now uses mDNS according to Wikipedia but how it uses that to launch im not sure

MagicMicky commented 4 years ago

So, I've been trying to play with wireshark to analyze what is sent by spotify on my computer when casting, it's only showing up as a TCP stream to port 8009

I pushed a bit more and tried to analyze the traffic when using the pychromecast spotify example. From that, I understood that the messages over port 8009 are most likely encoded via protobuf, with the definition in python here and in proto here

I haven't been able to decode the tcp messages from this though. I've tried using wireshark, but my knowledge is limited and it seems the packets are not detected as relying on protobuf.

On a completely different note, I've tried playing expiresIn argument of the setCredentials request sent to the Spotify app to log in. If I hardcode it to 30s, it will stop after 30s. I'm trying right now to put a expiresIn value greater than the default one of 3600 to see the impact; If I don't forget that I'm trying this out, I'll come back in around 1h with result (did it stop or not)

Edit: One additional thing I've tried is when I had the expiresIn set to 30s, trigger periodically a new setCredentials with a new expiresIn value (30s again); This had no impact and it was still expiring after the original 30s. I haven't found online reference to the API used as well, I'm not sure if it's a chromecast API, a spotify API, or maybe even the spotify chromecast app API.

Edit2: Changing the expiresIn to something greater than the token expiration time has not worked. The music stopped after 1h but the application didn't stop and the device became unavailable from spotify :/

Edit3: It seems I actualy missed it, but the protocol is wrapped in TLS and thus will be hard to be parsable via wireshark source 1 Cheers

fondberg commented 4 years ago

The payload of the messages are vendor specific and therefor not something which Spotify or anyone else would publish.

The only way is to do a proxy sniff, of a mobile phone playing something in a cast device for more than 60mins, which would terminate the TLS and forward. The problem here is that I don't think there exists a good tool for it unless you can do it in wireshark. It was a long time ago I did something with wireshark so this is not something I can advise on.

MagicMicky commented 4 years ago

Thanks for confirming it is indeed TLS encrypted

My fear is that the apps uses tokens with a longer expiration, and that the way to generate those are hidden and will be hard to duplicate.

I'll try to look at sniffing the encrypted traffic over the weekend. I'm far from an expert of wireshark, but I think it might be possible or the netsec community out there already developped similar tools.

fondberg commented 4 years ago

I didn't confirm that it is TLS. Maybe I misunderstood you.

Maybe the apps are using longer living tokens but if you start casting from the Spotify app on android many times it also stops after 60mins. I think the app needs to be open for it to refresh.

What we need is to decode the traffic between the app on android on android and the cast device.

It could be something simple like providing the refresh token in the setup

MagicMicky commented 4 years ago

So I spent a fair amount of time on Friday/Saturday trying to look into this, unfortunately it wasn't as successful as I hoped.

The protocol used by the android apps to communicate with the chromecast apps is called castv2. It relies on a protobuf based TCP connection wrapped in TLS, over port 8009. The protobuf messages can be found in pychromecast.

I managed to set a TCP over TLS proxy server on my rpi, that was intercepting all the traffic to my chromecast over the network. With wireshark, and using my proxy's tls private key i was able to decrypt all the traffic and display my decrypted protobufs (see protobuf over tcp in wireshark - needs to replace the dissector association with a tls.port instead of tcp.port). At this point I also set a dst NAT rule on my router to redirect all traffic to my chromecast to my proxy (mikrotik dst nat rule)

I was then able to display all the commands sent via pychromecast, but the traffic from the apps was blocked with a call over the namespace urn:x-cast:com.google.cast.tp.deviceauth, which wasn't happening with pychromecast.

Looking over the internet, and found a few good resource (node castv2 implementation, a castv2 protocol description, main discussion regarding deviceauth it seems that castv2 support a non mandatory authentication protocol. This protocol relies on 2 certificates interacting with one another and requires that the certificate being served to the client to be the one stored on the chromecast (mode details. Since I was using the proxy, it wasn't the case and thus the authentication was failing and the device was not proceeding to connect to the chromecast. Same between android, my chrome browser, and the macOS spotify application. It seems that there is a way to overcome the device auth on android, but it requires rooting and patching some android system apks, see Investigating Google Cast: Disabling device authentication on Android with Xposed. I'm not sure I'm ready to proceed with this as using xposed seems risky and I don't have any android backup/testing phone...

At the same time I also tried to play with the android spotify APK. If the calls such as setCredentials and potentially the refresh credentials one are handled at the spotify level, there should be traces of those in the APKs. Unfortunately, even after decompiling the apk with dex2jar and/or apktool, I didn't find a mention of those message. However, I found in a package called com.spotify.libs.connect.model something related to a getInfo and getInfoResponse, also used here in pychromecast

I as well tried to understand where the information regarding the setCredentials call come from. If some people managed to get the information regarding this specific calls existence, they might be aware of additional calls done for renewal. It seems that it comes from an article that was since then deleted (reddit thread, and I couldn't find the article anywhere). It seems that the author of the article is now hired by Spotify, so it might be a difficult way to investigate as well 🙃. If someone find the source article however, I'd be interested to understand how he reversed the protocol up till the setCredentials!

So far, still stuck and unable to define how android/desktop apps renew the tokens on the chromecast. Next chance will most likely be patch the casting libraries to skip authentication on android, but I'm not even sure to be able to do exactly what I'd want. Other way would be to keep looking into the decompiled APK but I didn't find much, maybe an older version of the app might...

I'm not sure I'll continue looking into this, but if anyone end up on this post looking to do the same, I can give a bit more details if needed!

Edit: ok, small update as I found the article and how the setCredentials information got out. The article links to the web player js library that contains specific chromecast message setCredentials and getInfo (from wayback machine). There's no mention of a refresh/update message though :/ Currently trying to cast from web seeing if it disconnects in about 1h. Edit2: It seems playback stops, so the web player doesn't seem to take care of renewal

fondberg commented 4 years ago

Great digging.

I used to work there until last year. Didnt work with cast though. What i know is that android and chrome have APIs for apps implementing cast. The payload can therefor be prioproetary after an app is launched on a cast device.

I listened to all events using pychromecast and received none.

I still think there is something simple like sending the refreshtoken alongside the access token. But the question is how

Jasperwolsing commented 3 years ago

Hi Niklas @fondberg , I want to help you with the 1 hour token issue.

I really what you made so far.

I have to take some time to read some history about what is investigated and not. Did someone try contacting Spotify about this?

Before using spotcast my chromecast was always playing, I just turned of my home cinema to turn off the music. Most of the times music was still playing After 24h while my phone who initiated the session was allready disconnected.

Regards, Jasper

logan893 commented 3 years ago

I too am experiencing this issue with Spotify streaming initiated from spotipy stops after at most 1 hour.

The expectation may be that the Chromecast devices will refresh the token, yet this isn't done in this case.

There seems to be two approaches for accessing Spotify. Authorization Code flow, where tokens can be refreshed, and Client Credentials flow, which seems to be the one used by spotcast and this doesn't appear to have any explicit refresh function.

Spotipy on the back-end can work with either type of token. https://spotipy.readthedocs.io/en/2.16.0/

I have the Authorization Code flow type token already configured for the regular Spotify integration. https://github.com/home-assistant/core/tree/dev/homeassistant/components/spotify

https://community.spotify.com/t5/Other-Partners-Web-Player-etc/API-Access-token-expired/td-p/4695256 https://developer.spotify.com/documentation/general/guides/authorization-guide/

fondberg commented 3 years ago

Hi. This has been raised a lot of times on the forum and the sad answer is that spotcast can't use any of the official oath ways because it needs a scope which is not available at all using their web api authentication.

That is why spotcast uses the browser login approach through spotify-token

dcnoren commented 3 years ago

Seems that one way around this would be a "watcher" service (automation) which monitors a) spotcast started, then b) spotify as source for 55 minutes, then essentially calls again the spotcast service. Downside is, of course, any playlist more than 55 minutes will not progress to later songs, unless on shuffle; and shuffle will be interrupted and re-shuffled, meaning some repeats from the first hour, etc.

That might help people who don't know why this is happening get some relief from the issue, if it was suggested on main page.

fondberg commented 3 years ago

Problem is that spotify app on the cast device needs to be stopped and started. But yes, an automation could do this

gmcmicken commented 3 years ago

Sorry, why can't we just hit https://api.spotify.com/v1/refresh with the refresh token, retrieve a new access token and pass this to pychromecast setCredentials?

https://developer.spotify.com/documentation/ios/guides/token-swap-and-refresh/

*edit

I have two suggestions to try. One, try calling refreshCredentials() instead of setCredentials() for manual refresh. and/or two, try setting expires to something like 3,300 to give the cast app time to do it refresh, it could just be failing before the timeout is triggered.

gmcmicken commented 3 years ago

Okay I spent some more time getting myself up to speed with this issue, thank you @MagicMicky and @fondberg for your work to investigate.

I tried sending refreshCredentials message in namespace urn:x-cast:com.spotify.chromecast.secure.v1, and unfortunately I do not get a response and the stream stops shortly after the expires time elapses. It is possible setCredentials will work but the stream needs to be stopped first? With this tool https://www.npmjs.com/package/chromecast-cli, I noticed the namespaces the current cast session support are:

urn:x-cast:com.google.cast.broadcast
urn:x-cast:com.google.cast.media
urn:x-cast:com.google.cast.cac
urn:x-cast:com.spotify.chromecast.secure.v1

and using the CAC tool https://casttool.appspot.com/cactool/ I can get all the stream info from the current session so that's cool.

My current idea for a workaround is to record the sessionId after launching and check this periodically to see if we're in the same session near the 1 hour mark - if we are in the same session we can pause the stream, re-load the cast app and transfer/pickup the stream again. I think this is possible with how spotify retains your position when pausing & closing an app or choosing a new playback device?

michaeltryl commented 3 years ago

I have a similar problem. Mine stops after approx. 10 minutes. However, I am not notified that it is playing on another device. Is there a solution?

Azouk112 commented 3 years ago

Hey all,

I am happy to try and support as I'm now hitting this 60 min timeout issue, reading the comments above did anyone get any further since the last update in November?

fondberg commented 3 years ago

@gmcmicken did you proceed any?

umutcelebi commented 3 years ago

Any updates? This is probably impossible to solve the issue i guess?

MagicMicky commented 3 years ago

Maybe I'm wrong but I think the changes done in https://github.com/fondberg/spotcast/pull/244 indeed has solved this; I didn't try that very carefully but that's the impression i've had lately

gmcmicken commented 3 years ago

Nice, good work @fondberg, I've been too busy with my kids (youngest is 2 months old) to contribute here, but have been following with my fingers crossed!