search

thlucas1 / homeassistantcomponent_spotifyplus

Home Assistant integration for Spotify Player control, services, and soundtouchplus integration support.
MIT License
47 stars 3 forks source link

Sonos Arc Control Errors (Lukas) #28

Open thlucas1 opened 3 months ago

thlucas1 commented 3 months ago

I get this response from my sonos speaker (using a ikea symfonisk):

Failed to call service spotifyplus.zeroconf_device_connect. ZeroconfResponse: SpotifyError="0" Status="103" StatusString="ERROR-UNKNOWN"

While trying to call this service:

service: spotifyplus.zeroconf_device_connect
data:
  entity_id: media_player.spotifyplus_lukas
  host_ipv4_address: 192.168.1.121
  host_ip_port: 1400
  cpath: /spotifyzc
  username: <canonical>
  password: <devicePassword>

Let me know if you have any other ideas... It would be sooo sick getting this to work

Note: using the .31 release from a few hours ago

_Originally posted by @s00500 in https://github.com/thlucas1/homeassistantcomponent_spotifyplus/discussions/22#discussioncomment-9864874_

thlucas1 commented 3 months ago

@s00500 Transferred comment from discussion to issue ...

Did another update to .34 today.

Again moving from sonos to my mac works fine, the other way around I get this error again

Failed to call service media_player/select_source. SAM0001E - An unhandled exception occured while processing method "PlayerTransferPlayback". SAM1003E - Spotify ZeroConf API returned an error status while processing the "Connect" method. Status: 103 - ERROR-UNKNOWN Message: "ZeroconfGetInfo: SpotifyError="0" Status="103" StatusString="ERROR-UNKNOWN" ActiveUser="" Aliases=[] Availability="" SupportedDrmMediaFormats=[] Version="2.9.0""

thlucas1 commented 3 months ago

Hi Lukas - Thanks for the update. I am still finalizing some changes with another user involving a Roku device; we will hopefully wrap that up today.

He was able to provide the mDNS activity between his desktop Spotify player and the Roku device. It really helped to see the data that was being exchanged between the 2 devices, and I was able to mirror that flow with my underlying SpotifyWebApiPython library. The data flow between the Spotify desktop player and the Spotify Connect device can vary by manufacturer. It's REALLY hard to code a solution when I don't have the physical device to experiment on. With that said ...

What I'm looking for is the mDNS activity for the getInfo, addUser, and resetUsers actions that flow between the Spotify Desktop player and the manufacturer Spotify Connect enabled device. A Wireshark trace will not reveal any password data (as it is encrypted in the blob data), but it will reveal your Spotify userid, IP addresses, Spotify Connect device ID's, etc.

For example, the below Wireshark trace led me to change the way the addUser is called. For my Bose Soundtouch gear, I only need to call it 1 time; for TCL / Roku devices, you have to call addUser to "wake" the device up, then call it again to actually login to the device. All of that was gleaned from the Wireshark trace.

It was also helpful to send a SmartInspect Trace log (SIL) file; however, that one will reveal detailed information about the SpotifyPlus integration and its configuration options that contain your userid and password. If it comes to me asking for that, I would suggest you create a temporary password for your Spotify userid that can be used for our testing. The SIL file can be encrypted so that it's secure to attach to a forum post.

WireShark Trace Example (sensitive data changed):

POST /zc HTTP/1.1
Accept-Encoding: gzip
Content-Length: 194
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Keep-Alive: 0
Host: 192.168.1.5:34814
User-Agent: Spotify/123700701 Linux/0 (PC desktop)

action=addUser&userName=1257898300&blob=&clientKey=&tokenType=default&loginId=canonicalvalue&deviceName=thinkpad&deviceId=eff383adad456422d43a77801a05ae587abf4095&version=2.7.1

Response to first addUser attempt:
HTTP/1.1 200 OK
Content-Type: application/json
Content-Security-Policy: frame-ancestors 'none';
Server: eSDK
Connection: close
Content-Length: 288

{"status":203,"statusString":"ERROR-INVALID-PUBLICKEY","spotifyError":0,"version":"2.9.0","deviceID":"48b677ca-ef9b-516f-b702-93bf2e8c67ba","publicKey":"KcUmGFl1ta+Vp7nNb4evrerbRpl1F8GKX9ShSrF79ip+M76LmtL6BADPoIRjA/3b5Wxh4p15t6Nb/dmybZfTnEDIQeHpSgx/Hl008cM+UZI3mCsq/VuAhM+/OjLnobKK"}

POST /zc HTTP/1.1
Accept-Encoding: gzip
Content-Length: 806
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Keep-Alive: 0
Host: 192.168.1.5:34814
User-Agent: Spotify/123700701 Linux/0 (PC desktop)

action=addUser&userName=myusername&blob=1DEmelcWwVZqFvwOUX7ZeR29L0K8M0Bm6%2Bveyh%2Bz3M7BSvcW%2B6orn2IXOZJy6bx2SDoyzPIi3BxMCLm6iMGIg94bwMqGqCm5Yz9D5EbNqphZTUMRfsGkEeUMs7E639It3eooPuXcu5Lnk3QuGvSz10SxVsgRBfkUBL878CDuSGp%2F5zSHwTGttJKqTlnaXDkxlpQsnTwwR0Fv6Ra%2BXI7uuomg5U2MZvYbW0aSLiF7l%2BeR643WbWgN7JoTm02KVYeVzEey%2BchLU1jxf4DYNLnaZldEBjRGLx%2BFN8ijnYK8wmxLKGM%2BolYbGSYdKUk9ccPEJP0MgXPhuJZ6999hmH7gl4c8QEPnRQpRVRUDAHxSAcVlI91eNE8sAEmFBYshpJ8kan3FDoempBhld%2FIrDnOzVtTAa2c%2F4dNSXO%2FHAiecdhGJWCBJkwzuuH%2BYnGgBzD9%2B&clientKey=hkA0nU0RlHvAIR8YfHTpx4kX%2BwLy%2FuJ36CvA3mngnflj%2FwPLjbKZlbihPfcbaP1bY7KrHja%2ByebIxjIhYvaWhYITSwe7r9ynzLpAWfxTRzK5QKh8r8t8YdDbu7mdBpl4&tokenType=default&loginId=canonicalvalue&deviceName=thinkpad&deviceId=eff383adad456422d43a77801a05ae587abf4095&version=2.7.1

Response to second addUser attempt:
HTTP/1.1 200 OK
Content-Type: application/json
Content-Security-Policy: frame-ancestors 'none';
Server: eSDK
Connection: close
Content-Length: 53

{"status":101, "statusString":"OK", "spotifyError":0}
thlucas1 commented 3 months ago

@s00500 Hi Lukas.

I wrapped up the issue with the other user, and am ready to tackle the issue with your Sonos Arc. With that said, can you please update to the latest SpotifyPlus v1.0.38, and try your test scenario again? There have been a bunch of changes with return-code processing between v1.0.32 and v1.0.38.

Please let me know when you get a minute. Thanks - Todd

s00500 commented 3 months ago

OK, retested with .38, no change in behavior unfortunatly...

I have a wireshark dump ready (limited to port 1400) How can I best send that to you ? I can encrypt it using age but it seems you have no publically viewable public keys on github ? Let me know how to best proceed

https://github.com/thlucas1.keys

thlucas1 commented 3 months ago

I’m a Windows OS kind of guy, so am not familiar with the age tool. Would you be ok with just attaching it to the forum post? Or sending it to my private email address?

Other than IP addresses and your Spotify loginID, it should not contain any visible password credentials or other PII data. The blob that is passed in the addUser request is encrypted using a public / private key exchange.

s00500 commented 3 months ago

any chance you can let me know that email address ? =D

s00500 commented 3 months ago

ok, nevermind, took the one from the commit log

thlucas1 commented 3 months ago

Got it, thank you.

Based on the getInfo responses, you have 4 Spotify Connect devices: A3Main, LBsRoam, Workroom, and Living Room.

The getInfo JSON response for each indicates a tokenType="authorization_code". I have not encountered this token type before in my Spotify Connect Zeroconf protocol research.

With that said, what we need to do is to capture the mDNS protocol data flow between the Spotify player (preferably the desktop version so you can Wireshark trace it) and the Sonos device. I need to see what the addUser requests look like, and try to mimic the flow.

To do this, pick one of the 4 devices (e.g. Workroom). We will use the IP address of that device (e.g. 192.168.1.121) in the steps below.

The following instructions (generic format using my device) can be found on the SpotifyPlus Wiki documents on GitHub.

Spotify Connect Device Tracing

The following walks you through how to trace Spotify Connect activity that takes place between the Spotify player (e.g. Spotify desktop app) and the Spotify Connect device (e.g. Bose Soundtouch, Sonos, Yamaha, etc).

Requirements

The following are required in order to capture Spotify Connect device activity:

Capture Login Flow

Use the following steps to capture the Spotify Connect logon flow.

1) Ensure that the Spotify desktop application is shut down; once we start it back up, it should re-discover any Spotify Connect devices on the network as well as log them in.

2) Start the network trace, using your tool of choice (e.g. WireShark, etc). You can limit the trace to the IP addresses of the Spotify Connect device and the Spotify Desktop application, or capture everything and filter the results later.

3) Issue a disconnect request to the device.
This will cause the device to logout from Spotify, and remove the device from the Spotify player's active device list. You can use the curl command-line tool to do this, using the Spotify Connect device endpoint URI and action=resetUsers parameter as a POST request.

Example, with response:
```
C:\Users\thluc>curl -X POST http://192.168.1.121:1400/spotifyzc -d "action=resetUsers"
{"status":101,"statusString":"OK","spotifyError":0}
```

4) Start the Spotify desktop application.
This should re-discover any Spotify Connect devices on the network, as well as log them in.

5) Stop the network trace, and save the results.

I'm specifically looking for the addUser requests and responses. We should be able to mimic these via the SpotifyWebApiPython library to control the device.

s00500 commented 3 months ago

Hey @thlucas1, thanks for clear instructions, I have sent you a new capture that contains the addUser response (Meanwhile: big thanks for your amazing response time!!!) Let me know if you need anything else

thlucas1 commented 3 months ago

FYI - just released a new version of the SpotifyPlus integration .

[ 1.0.39 ] - 2024/06/28

It appears that the clientKey value is not used when the token type is authentication_code: HTML Form URL Encoded: application/x-www-form-urlencoded Form item: "action" = "addUser" Form item: "userName" = "1156377975" Form item: "blob" = "AQB8ay46qJJ8WWL ..." Form item: "clientKey" = "" Form item: "tokenType" = "authorization_code" Form item: "loginId" = "865362db394e05690a9faab8af41d883" Form item: "deviceName" = "LBsMBP" Form item: "deviceId" = "5f8bab4eecfbb98b8e68825467a3907e3e527a8e" Form item: "version" = "2.7.1"

I made a change to the underlying SpotifyWebApiPython code to account for this, and hopefully it will work. I say hopefully, because I don't know if the encrypted blob contains the same information for the authentication_code token type as it does for the default token type. This change should confirm that though.

Give the SpotifyPlus v1.0.39 a try. You can use the following script to test various Spotify Connect related services. You might also run a Wireshark trace while you are executing this, in case it fails I will need to see the trace. I set it up to point to your Workroom device - adjust IP addresses and device name if you want to test it with another device. Also adjust the following settings to your environment:

s00500 commented 3 months ago

Hm... this still seems to result in the same... when trying to switch source now I get this:

Failed to call service media_player/select_source. SAM0001E - An unhandled exception occured while processing method "PlayerTransferPlayback". SAM1003E - Spotify ZeroConf API returned an error status while processing the "Connect" method. Status: 103 - ERROR-UNKNOWN Message: "ZeroconfGetInfo: SpotifyError="0" Status="103" StatusString="ERROR-UNKNOWN" ActiveUser="" Aliases=[] Availability="" SupportedDrmMediaFormats=[] Version="2.9.0""

When running your script I already get this 103 status in the "resolve deviceID" request already... the other requests seem to not work or also result in that...

thlucas1 commented 3 months ago

Can you start a Wireshark trace, run the script, then send me the trace file?

My fear here is that the encrypted blob contains different information when TokenType="authentication_code". If that is the case, then I may not have a solution as I don't know what the Spotify SDK is expecting in that blob value for that token type.

Let me take a look at the trace first though, to see if anything jumps out at me.

UPDATE - I just thought, how are you going to run a trace if you're not on the HA machine? LOL
Let me do some more searching to see if I can find out what's in the blob for token type "authentication_code"

thlucas1 commented 3 months ago

Another possibility is to setup a SmartInspect trace - more info on that can be found on the SpotifyPlus wiki Tracing HA Integration Components page. This will give me detailed information on the Spotify Connect process flow as well, if you can't get a Wireshark trace.

Full disclosure, the SmartInspect trace file will contain detailed information about the SpotifyPlus integration - this includes your Spotify Username and Password that is stored in the configuration options. I can walk you through how to remove the log entries that contain sensitive information if you like, though it would require that you install the SmartInspect trace viewer that requires Windows OS. More info about SmartInspect Tracing can be found on the SmartInspect Logging Configuration wiki page.

Let me know if you would like to proceed with that.

thlucas1 commented 3 months ago

I forgot to ask / mention this in my previous replies, but have you set the Spotify Connect related properties in the SpotifyPlus Integration Options form? See example form below.

If not, you will want to set the Spotify Connect LoginId, Username, and Password values, and run the test script again.

image

s00500 commented 3 months ago

Ok, was thinking the same when reading your first text haha

I prefer to stick to wireshark tough, I will setup port monitoring for my homeassistant instance so I can do a proper capture tomorrow...

Finally yes, I just checked. I only had the first field empty, but changing it did not help at all

I have also found this, but it might be very unrelated: https://github.com/fondberg/spotcast/issues/285

thlucas1 commented 3 months ago

I took a look at the SpotCast article you mentioned. The last reply pretty much says it all:
This only works when the Sonos device is already in Spotify Connect mode.

We already have the deviceId via the getInfo results, so that's not a problem.

The issue is that the device has issued a logout of the Spotify Connect, which causes it to drop from the (active) device list. What we are trying to do at this point is to re-activate the device (force it to log back in basically) to Spotify Connect, and be added back into a Spotify player's active device list.

I am not sure I agree with the comments regarding "Sonos uses a layer incompatible with Spotify API" and "Since Sonos uses a different API". Sonos clearly implements the Spotify Zeroconf API, as it is responding to getInfo and addUser requests; the problem is that we don't know how to "talk" to the addUser API to get the device to log back into Spotify.

The tokenType=authorization_code leads me to believe there is some sort of token processing going on, but how to interface with that is the question. I am betting there is something different about the blob parameter that is being passed; rather than a username and password (like tokenType=default uses), it could have something else - what that is I do not know. Which is where the network or SmartInspect trace comes into play - maybe it will provide a clue.

I just found out from this past experience with TCL Roku, you have to make two addUser requests to cause the device to re-login. The initial request has the availability set to NOT-LOADED, it also returns a new publicKey value; when that occurs, you have to issue another addUser with the newly returned publicKey and poll the device to wait for it to start up (availability becomes ""). Once that happens the device is then active, and starts showing up in the active device list. Note that if the device is already logged in, you only have to issue one addUser request. This is a TCL-specific scenario I believe; my Bose SoundTouch devices (fairly old) only require one addUser request to activate the device.

I'm hoping the Sonos stuff is something similar to the TCL processing.

thlucas1 commented 3 months ago

Forgot to ask … can you execute just the “Test Connect” step of the script. Curious to see what the response is for just that step.

thlucas1 commented 3 months ago

Found a few links on the web that talk about Sonos internals that might be worth pursuing to see if the device logs any Spotify Connect details.

Sonos Log Files

found this article that talks about log files that are kept on the Sonos device itself. I don't know if they would be useful, or even if they are still accessible (7 yr old article), but it might be worth a look.

Here's the article text marked as the solution to the OP's question:

Go to http://Player_IP_address:1400/support/review. Choose an IP from About My Sonos System. Your workroom device would be: http://192.168.1.121:1400/support/review

All you see there is for Sonos internal use only, and is undocumented, but we've figured out a few things over the years.

The main application log is at /opt/log/anacapa.trace. Timestamps use the offset internal clock. The Linux system log is at /bin/dmesg. Timestamps are seconds from boot.,

Of potential interest is the RF information at /proc/ath_rincon/status, though the Network Matrix provides this in a more concise and digestible format. If the system isn't in SonosNet mode (aka 'BOOST Setup') then the matrix is meaningless.

/proc/ath_rincon/phyerr is a histogram of physical errors on the RF over the last 20 minutes.

Hidden Sonos Interface

I also found this article about a hidden Sonos interface. Might be worth checking into.

The status screen can be used to gain insight into the player setting, its hardware, and its environment. It is available at the following URL:

http://192.168.1.121:1400/status The screen displays a large collection of submenus that you can explore at your leasure. Many of these menus such as 'dmesg', 'netstat', and so on will be familiar to Unix users since they display the result of the corresponding command (Sonos players run the Linux operating system internally).

s00500 commented 3 months ago

Ok, something interesting is happening all script steps.pcapng.zip here. the response from the device is kind of broken... this log all the way to HA but only seeing it in the traffic I really believe it, check these captures:

What I have done here is basically just run the script step by step. Since resetUsers fails on the "Connect" step the addUSer is never fired. Seems the device only responds with a single bracet

rincon

Here is the status from my sonos interface page, nothing super interesting there unfortunatly...

s00500 commented 3 months ago

(the wireshark capture is performed in between the Raspberry running HA and the rest of the network)

s00500 commented 3 months ago

Here is again the exact response from the device once the addUser request fires (when doing a source switch on the player entity)

HTTP/1.1 500 Internal Server Error
Content-length: 80
Content-type: application/json; charset=utf-8
Server: Linux UPnP/1.0 Sonos/79.1-54060 (ZPS33)
Connection: close

{"status":103,"statusString":"ERROR-UNKNOWN","spotifyError":0,"version":"2.9.0"}
thlucas1 commented 3 months ago

I am looking at the Wireshark trace now. I should have asked you to run those steps individually; sorry for not stating that up front - it sounds like you figured that out though.

Let's analyze the script steps individually, across separate replies if that's ok. Let's start with the Test Disconnect step. The Test Disconnect step looks like it is working, but does not return a JSON response (which is odd). It is returning an HTTP status of 200 (OK), but only a "}" response (not valid / incomplete JSON). It should be returning something like this:

{
  'status': 101, 
  'statusString': 'OK', 
  'spotifyError': 0
}

I want to verify that the Sonos Device is actually being disconnected, so let's run the Disconnect step again if you would using the following How To Test guidelines:

How To Test

You can also test it via a curl POST command from command-line. Would be interested to see if that returns a JSON response:

C:\Users\thluc>curl -X POST http://192.168.1.121:1400/spotifyzc -d "action=resetUsers"
{"status":101,"statusString":"OK","spotifyError":0}

Did the device disappear from the Spotify player active device list?

s00500 commented 3 months ago

So I did this by checking in a webbrowser, as this is the only place where I can only see these devices fresh after activation.

the resetUsers certainly worked, the response was weird again though:

❯ curl -X POST http://192.168.1.121:1400/spotifyzc -d "action=resetUsers"
}%

Procedure I did:

s00500 commented 3 months ago

Other things I found: https://johnnycrazy.github.io/SpotifyAPI-NET/docs/authorization_code/

AuthCode and loginid are changing on each addUser request (I did the reset a few times...)

I also see one request to accounts.spotify.com when just before the addUser request, but I bit my teeth out trying to capture it unfortunately.... seems like the spotify clients use proper certificate pinning here.... I was trying to capture the request with charles proxy and see if I could catch it... but no luck on that

thlucas1 commented 3 months ago

Regarding the resetUsers response, that is a bug in the Sonos implementation of the Spotify Connect Zeroconf API. Based upon what you observed, it seems to be functioning as expected (e.g. device disappears from active device list) which is good.

I am going to add some defensive code in my API though to account for it. If a JSON response was not returned, then it will treat the http status code as the response code; if it's not a 200, then it will raise an exception.

thlucas1 commented 3 months ago

Regarding the SpotifyAPI-NET link you sent, that appears to be the AuthorizationCode for the Spotify Web API. I am already aware of that process, and use it in my SpotifyWebApiPython package, which is used by the SpotifyPlus integration. I wrote that library as well, so am well versed in how that process works.

For the request you saw to the https://accounts.spotify.com/authorize endpoint, was that coming from the Sonos device? Or was it coming from the Spotify player / desktop client? If it was from the Spotify player / desktop client then that is normal activity.

If it was from the Sonos device, then it means that they are using OAuth token authentication directly from the device. If that is the case, then we might be out of luck with trying to figure this out; the OAuth uses a clientId and clientSecret to authenticate to Spotify, of which those values are known only to the Sonos developers and the device. It's exactly like the process that Spotify and SpotifyPlus integrations use to access the Spotify Web API, except they are using the clientId and clientSecret of YOUR Spotify developer application. Since we don't know the client id or secret value, we won't be able to connect to the proper Spotify OAuth application.

You might try the following sequence to ensure that you capture the correct protocol exchange: 1) exit all Spotify player applications (desktop, web, mobile, etc). 2) start a Wireshark trace on the desktop machine that will be running the Spotify desktop application. 3) use the resetUsers command to drop the device from the active device list. curl -X POST http://192.168.1.121:1400/spotifyzc -d "action=resetUsers" 4) start the Spotify player application on the desktop machine. 5) verify that the device you dropped in step 3 is back in the Spotify player active device list. 6) stop the Wireshark trace and diagnose.

I'm hoping the Spotify desktop player will use the Spotify Zeroconf API addUser action to communicate with the dropped Sonos device to reactivate it. I would think it would be using the Zeroconf API to do that, as it supports the resetUsers and the getInfo endpoints. The question is, what does that protocol exchange look like, and what's in the blob?

s00500 commented 3 months ago

Hey @thlucas1

I might have not fully communicated that well, yes certainly the desktop or mobile device talks to the web API.

I observed that request using Charles proxy, a little tool that allows capturing all outgoing traffic from your computer. The nice thing about Charles is that it can supply a ssl root cert that can be installed into the trust store of your device. This normally allows for seeing also outgoing TLS traffic in its plain form. But the Spotify desktop and mobile apps seem to expect a exact certificate finger print (pinned) so I could not perform this kind of analysis

I saw that you are the maintainer of the other library too :-) nice work

So the remaining question is, how to get this "authorization_code" from the API that you need to send to the device...

Can you explain a bit how that normally works? I assume you are using oauth2 to get a accesstoken from the webapi and then send it to the speaker ? The only difference here is the tokenType?

Any way you can request that in the request to spotifys web api?

thlucas1 commented 3 months ago

I don’t think the authentication flow works the same for the Sonos device as it does the web api.

For web api, you performed the Spotify OAuth exchange when you setup the integration application credentials. You entered your Spotify developer application client I’d and secret, then it redirects you to Spotify web to login, then it returns a authorization token. That token is then cached in HA Application Credentials data store and used by the Spotify and SpotifyPlus integrations. Note that the token is only valid for 60 minutes, and is just constantly renews after that.

Since the Sonos is a “headless” device, I am assuming it goes thru the same type of process, but it does it when you go through their app to “Add a Service” (in this case Spotify). You will login and give the Sonos Spotify Developer Application access to your user account. I assume the Sonos cloud will then cache the authorization token and associate it with your username.

now for the theoretical part …

After that, you specify the Spotify username you want to access for the device in the Spotify ZeroConf AddUser request, then the device uses the authorization token for that user to control the device.

I don’t think the addUser blob contains the authorization token, as it’s too small of a length (tokens can be larger), plus there is no way to access the Spotify web to get a new token since it’s a “headless” device.

I think we just need to figure out what’s in the blob in order to get it to work. I think it’s possible, as the addUser request is encrypting it with a shared secret. I am working on how to do that now. I have to add some SmartInspect trace statements to the blob builder code to research it though.

Will keep you posted.

s00500 commented 3 months ago

Ok thanks for the update

I just want to note again that I think you got this slightly wrong...

There is never a step where you login via the sonos app, the sonos device has no clue of the spotify user.

I still believe this is how it works:

Desktop client finds sonos on zero conf and fetches get info

Finds out it needs authorization_code

Desktop client contacts accounts.spotify.com to fetch such an authorization_code. (This request is authenticated using normal token created via oauth of course)

Finally desktop client sends this temporary authorization code and a temporary loginID to the sonos in the "addUser" request

In this case the authorization_code blob might not even be anything valid at all, just a token created and then verified by spotifys server

If we find how to get it from the api we can do the same, without the need of knowing what it actually is

This is at least my theory of what os going on

One strong indicator for my theory is that wen you resetDevice and then block that specific request from the desktop client to the accounts.spotify.com host The desktop client never successfully transfers playback. I have tested this on my setup

thlucas1 commented 3 months ago

I have been wrong before, so no offense there.

Regarding your comment: There is never a step where you login via the sonos app, the sonos device has no clue of the spotify user.

There has to be some step in the Sonos App where you add the Spotify user account(s) that you want to access, and it prompts you to login to Spotify or grant it access to a Sonos Application. Otherwise it would not know who you are and would not be able to pull your specific content (playlists, favorites, etc). The authorization token is generated at that time, and stored somewhere - probably in a Sonos cloud service. The token is then repeatedly refreshed when needed, so that the user never has to re-authenticate to Spotify.

The https://accounts.spotify.com/authorize url is used to request user authorization permission for an authorization token. The https://accounts.spotify.com/api/token url is used to request or renew a Spotify authorization token.

Do you see either of those 2 url's in your trace logs anywhere?

Regarding your comment: One strong indicator for my theory is that when you resetDevice and then block that specific request from the desktop client to the accounts.spotify.com host The desktop client never successfully transfers playback.

The Spotify Web API '/me/player/devices' endpoint will contain the devices that the user is currently logged into and has control of. The Sonos device would have to have a user context established in order to retrieve that list, as well as issue the transfer of playback to a device.

I guess I am still not sure how it works, but will hopefully know more when I find a way to decode the blob.

thlucas1 commented 3 months ago

Forgot to mention ... I have a OAuth2 Authorization Protocol wiki document that I put together that explains the OAuth2 flow if you want to have a look at it.

thlucas1 commented 3 months ago

Can you try the following service calls (from the Developer Tools \ Services form) and let me know what the response is that you get? Just change the YOUR_SPOTIFY_PASSWORD to your Spotify password.

I don't think the loginid parameter value was specified in the Script Test Connect step. Just wondering if that might make a difference.

This one will test the non-canonical username value, along with the loginId value.

service: spotifyplus.zeroconf_device_connect
data:
  entity_id: media_player.spotifyplus_lukas
  host_ipv4_address: 192.168.1.121
  host_ip_port: 1400
  cpath: /spotifyzc
  loginid: 865362db394e05690a9faab8af41d883
  username: "1156377975"
  password: "YOUR_SPOTIFY_PASSWORD"

This one will test the canonical username value, along with the loginId value.

service: spotifyplus.zeroconf_device_connect
data:
  entity_id: media_player.spotifyplus_lukas
  host_ipv4_address: 192.168.1.121
  host_ip_port: 1400
  cpath: /spotifyzc
  loginid: 865362db394e05690a9faab8af41d883
  username: 865362db394e05690a9faab8af41d883
  password: "YOUR_SPOTIFY_PASSWORD"
thlucas1 commented 3 months ago

Can you try the following service call (from the Developer Tools \ Services form) and let me know what the response is that you get? Prior to executing the call, there needs to be a Spotify track (or playlist) playing on the 192.168.1.121 Sonos device.

I'm looking to see what it returns for the listed results while a Spotify track is actively being played on the device.

service: spotifyplus.zeroconf_device_getinfo
data:
  entity_id: media_player.spotifyplus_lukas
  host_ipv4_address: 192.168.1.121
  host_ip_port: 1400
  cpath: /spotifyzc
  version: 1
  use_ssl: false

I want to see what it returns for the following properties: (example values listed for my device): ... SpotifyError: 0 Status: 101 StatusString: OK ResponseSource: null AccountReq: DONTCARE ActiveUser: xxxxxxxxxxxxxxx Availability: "" ClientId: 79ebcb219e8e4e9a892e796607931810 DeviceId: 30fbc80e35598f3c242f2120413c943dfd9715fe

s00500 commented 3 months ago

Here is the full response from the device while a track is playing

user_profile: country: AT display_name: Lukas email: lukas@lbsfilm.at id: "XXXX" product: premium type: user uri: spotify:user:XXXX result: SpotifyError: 0 Status: 101 StatusString: OK ResponseSource: null AccountReq: null ActiveUser: "" Aliases: [] Availability: "" BrandDisplayName: Sonos ClientId: 9b377073ea334637b1406f329ce005de DeviceId: 4f3990f476a5eff1772c93aa8272ddc5480a4124 DeviceType: SPEAKER GroupStatus: NONE LibraryVersion: 3.199.414-gea87b026 ModelDisplayName: Bookshelf ProductId: 1233 PublicKey: >- te61esVMfBfddj8AoNVs+/QfdlN1pvPbfBojKMmVbRQwiHMK9kMbjJV2iY4oRIwWR8gPwagwSZFsCgEXZbmAUTpBDz+7cKEmJzuK2ozkd1MCG+Oy4tRysnLnCoV/pZZv RemoteName: Workroom ResolverVersion: "0" Scope: streaming SupportedCapabilities: 3 SupportedDrmMediaFormats:

thlucas1 commented 3 months ago

@s00500 Thanks Lukas. I find it unusual that the ActiveUser is not populated with a username or canonical userid while a Spotify track is playing for the user account.

Can you try the following service calls (from the Developer Tools \ Services form) and let me know what the response is that you get? Just change the YOUR_SPOTIFY_PASSWORD to your Spotify password.

I don't think the loginid parameter value was specified in the Script Test Connect step. Just wondering if that might make a difference.

This one will test the non-canonical username value, along with the loginId value.

service: spotifyplus.zeroconf_device_connect
data:
  entity_id: media_player.spotifyplus_lukas
  host_ipv4_address: 192.168.1.121
  host_ip_port: 1400
  cpath: /spotifyzc
  loginid: 865362db394e05690a9faab8af41d883
  username: "1156377975"
  password: "YOUR_SPOTIFY_PASSWORD"

This one will test the canonical username value, along with the loginId value.

service: spotifyplus.zeroconf_device_connect
data:
  entity_id: media_player.spotifyplus_lukas
  host_ipv4_address: 192.168.1.121
  host_ip_port: 1400
  cpath: /spotifyzc
  loginid: 865362db394e05690a9faab8af41d883
  username: 865362db394e05690a9faab8af41d883
  password: "YOUR_SPOTIFY_PASSWORD"

Thanks

s00500 commented 3 months ago

First call results in

Failed to call service spotifyplus.zeroconf_device_connect. ZeroconfGetInfo: SpotifyError="0" Status="103" StatusString="ERROR-UNKNOWN" ActiveUser="" Aliases=[] Availability="" SupportedDrmMediaFormats=[] Version="2.9.0"

Second call in

Failed to call service spotifyplus.zeroconf_device_connect. ZeroconfGetInfo: SpotifyError="0" Status="103" StatusString="ERROR-UNKNOWN" ActiveUser="" Aliases=[] Availability="" SupportedDrmMediaFormats=[] Version="2.9.0"

I see that as another indicator for my theory from a few comments up. The sonos device is not keeping a login for these sessions, it just uses a temporary access code created by the desktop/mobile apps. This is also why it has to be reacivated every time

thlucas1 commented 3 months ago

Thanks for the information.

At this point, your theory makes more sense as there is no ActiveUser established. For every other device type I have seen, there is an ActiveUser established at some point in the process; but then again, the TokenType is default for those.

I still hold true to parts of my theory regarding the blob data. There has to be something in the encrypted blob of the addUser action that lets the Sonos device know who the user is. I have not been able to decrypt a blob without the corresponding private key (Diffie-Hellman cryptographic functions).

For reference, here is the blob layout for tokenType=default:

# create the blob data that will be sent to the device.
blob = bytearray()
# 'I'
write_int(0x49, blob)
# username
write_bytes(self.credentials.username, blob)
# 'P'
write_int(0x50, blob)
# auth_type
write_int(self.credentials.auth_type, blob)
# 'Q'
write_int(0x51, blob)
# password
write_bytes(self.credentials.password, blob)
_logsi.LogBinary(SILevel.Verbose, "Blob Data, Unencrypted Bytes", blob)
# padding
n_zeros = BlobBuilder.AES_BLOCK_SIZE - (len(blob) % BlobBuilder.AES_BLOCK_SIZE) - 1
blob.extend([0] * n_zeros)
blob.append(n_zeros + 1)
_logsi.LogBinary(SILevel.Verbose, "Blob Data, Unencrypted Bytes (with padding)", blob)

The above results in a blob that looks like this (before encryption of course): image

At this point I think I have exhausted all of my possibilities. I don't know who to reach out to for specs on the blob structure for the tokenType=authorization_code. If I had that it might provide more clues as to what (exactly) I need to provide. I will try to post a request on the Spotify Developer forum, but I don't think that will go anywhere.

Any other thoughts or ideas?

thlucas1 commented 2 months ago

This conversation on TokenType authorization_code looks promising; it's under the spotify-connect repository issues. Unfortunately, the commenter that was making progress has not commented since Feb 2023.

This would be much easier to diagnose if I had a Sonos device to test with. As I do not have a Sonos device, I am going to place this on the back burner until I can get access to one for direct testing. I appreciate your help thus far on this, but think we have exhausted what we can do without me having an actual device to test with.

Please reach out if you find or hear of a way to do this, and we will revisit.

thlucas1 commented 2 months ago

@s00500 Hi Lukas - I have the Symfonisk configured, and the Spotify desktop app recognizes it. I can play content on it via the Spotify desktop player, and that works fine.

The problem is that I cannot control the device via the Spotify WebServices API. It always returns a 403 - restricted device error when I try to issue player commands (e.g. pause, resume, next track, etc). The same result is returned no matter which control method is used (Spotify Developer WebService API portal, my SpotifyWebApiPython library, Home Assistant SpotifyPlus integration, etc).

I am not sure how the Spotify Desktop player handles the request - is it interfacing directly with the Sonos Control API to perform these functions? That would be my guess.

It was my assumption that you were just having issues with the Symfonisk device disappearing from the Spotify Connect device list. Are you having issues controlling the device as well? e.g. Pause, Resume, Next Track, etc?

s00500 commented 2 months ago

Hey Todd,

No never had issues with that, once I have connected I can control the playback session using any spotify device, phone, webbrowser, streamdeck integration and also spotifyplus via HA...

I had some errors once but I think they resolved by some powercycles...

How exactly did you test it?

s00500 commented 2 months ago

Ok, I quickly gave this a test again,

Connected to sonos, started playback, killed any spotify app or desktop client in my network

I can not control it using spotify plus then, getting error restricted device

BUT I can use the spotify browser client, so it seems like there is still a way and no local communication is required...

s00500 commented 2 months ago

The spotify webplayer seems to send the pause command like this:

POST https://gew4-spclient.spotify.com/connect-state/v1/player/command/from/XXX/to/XXX

{ "command": { "logging_params": { "page_instance_ids": [], "interaction_ids": [], "command_id": "XXX" }, "endpoint": "pause" } }

thlucas1 commented 2 months ago

The pause command from the Spotify Web player that you mentioned is sent using the Spotify Web Playback SDK to control the player.

Per the Spotify documentation ... The Web Playback SDK is a client-side only JavaScript library designed to create a local Spotify Connect device in your browser, stream and control audio tracks from Spotify inside your website and get metadata about the current playback.

The SpotifyPlus integration uses the Spotify Web Services API (a server-side library) to control playback.

One would think that the Web Playback SDK and the Web-Services API would perform the same when it comes to controlling the player, but (obviously) they don't.

As for test scenarios ...

Spotify Connect Zeroconf resetUsers Request

I sent the request via Postman to the Symfonisk Office device name (192.168.1.91) , and it returned with a status 200 OK. The request was sent while a track was playing in the Spotify Desktop Player.

Spotify Connect Zeroconf getInfo Request

The ActiveUser property always returns an empty string.

Sonos Device Information

I found a bunch of links that display Sonos information from the device:

Sonos Control API

Found some helpful links for how to control the Sonos device via the Sonos Control API:

I was able to issue a call to the Sonos Control API to mute / unmute the device using SOAP web-services (via Postman).

<?xml version="1.0" encoding="utf-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
  <s:Body>
    <u:SetMute xmlns:u="urn:schemas-upnp-org:service:RenderingControl:1">
      <InstanceID>0</InstanceID>
      <Channel>Master</Channel>
      <DesiredMute>0</DesiredMute>
    </u:SetMute>
  </s:Body>
</s:Envelope>

This might provide a way to get other information from the device regarding the Spotify music service (current status, transport control, etc). I am hesitant to go down this route though, as the SpotifyPlus integration specifically uses the Spotify WebServices API to control Spotify playback. There would be special calls into the Sonos API to control Sonos devices.

Just some initial thoughts thus far.

thlucas1 commented 2 months ago

Apparently the Sonos restricted device issue has been known about since 2018, per this article.

devices endpoint does not find my Sonos One pair, even during playback. player endpoint does, though with a null id:

"device": {
    "id": null,
    "is_active": true,
    "is_restricted": true,
    "name": "Sonos Véranda",
    "type": "Speaker",
    "volume_percent": 0
  }
s00500 commented 2 months ago

Hm very interesting... I can control playback via the sonos integration on homeassistant though... but still no way to move the current session...

Meanwhile I tried to get Frida to work on macOS... got it running but the script I used failed anyways... if anyone has good tips on bypassing ssl pinning let me know... else I probably need an android device to do so...

thlucas1 commented 2 months ago

I believe the Sonos integration is using the SoCo api to interface directly with the Sonos Control API webservices to control the Sonos player. The SpotifyPlus integration uses the Spotify WebServices API to control the player. It seems like the Spotify WebServices API implements SOME of the functionality to interface with Sonos (e.g. playback status, current track playing, etc), but not other features related to the play transport controls (e.g. play, pause, next track, etc).

thlucas1 commented 2 months ago

Hi Lukas,

Some more observations regarding Sonos behavior with Spotify ...

In order to implement Sonos support, I would need to add Sonos-specific function calls to all pieces of code where I call the Spotify WebServices API. Something like:

if device_is_sonos:
   call sonosapi_play()
else:
   call spotifyapi_play()

This type of logic would need to be done for transport functions (pause, play, next track, previous track, seek, volume, mute, set repeat, set shuffle), player status (current album, track, artist, cover image, position, duration), group / zone functions, etc.

That is basically a rewrite of the entire player.

s00500 commented 2 months ago

Hey Todd, Interesting, but I dont think this is necessarily needed, as one can rely on the sonos integration in homeassistant when such players are used.

What still would be nice would be allowing a transfer of the session.... as I know of no other way to do this at the current point...

thlucas1 commented 2 months ago

Hi Lukas,

FYI - just released a new version of the SpotifyPlus integration

[ 1.0.41 ] - 2024/07/18

This should give you the ability to transfer playback to another Spotify Connect device, including Sonos devices. It's only been tested with the Sonos Symfonisk, so let me know if it works with your other Sonos device types.

To transfer playback, use the Player Transfer Playback, specifying the Spotify Connect device name to transfer to. Like so:

service: spotifyplus.player_transfer_playback
data:
  entity_id: media_player.spotifyplus_lukas
  device_id: "Sonos-Office"
  play: true

Note that this does not address the tokentype=authorization_code issue that we have been discussing prior. I took a different approach to the problem, and came up with this.

I also put together a Spotify Connect Brand Notes page that documents some of the quirks and oddities that I have ran into trying to control Sonos Spotify Connect devices.

Give the new version a try when you get a chance, and let me know how it works out for you.

Thanks!

s00500 commented 2 months ago

Hey Todd,

This sounds AMAZING

I am still out on travel till end of next week, but I will certainly let you know when I am back home

Thanks a lot !!! 👍👍👍