API Changes

[kingosticks]

Soundcloud devs seem to be alive! They've made / are making some API changes which look like they will impact us. The API docs have moved here. The old endpoints still seem to work as of now. They also seem to be doing something to pagination and it sounds like the default response format is going to change.

[djmattyg007]

How does this affect the PRs that @Laurentww has submitted recently?

[kingosticks]

I've only looked at the rate limit one. Only the image one seems to hit the official API and that's restricted to the tracks endpoint which I don't think changed. Looks like it's mostly the /me/foo endpoints.

[Laurentww]

Adding images (#121) still uses the official API, as the standard app client id doesn't return rate errors here. Though, #121 could be improved as well by first attempting with the publicly obtained client id and reversing to the standard app client id on failure.

As far as I can see, no pagination calls are made and the current API call uris are up to date.

[kingosticks]

More api changes coming https://developers.soundcloud.com/blog/security-updates-api

[dreamlayers]

It seems the security related changes prevent mopidy-soundcloud from working now, as explained in https://stackoverflow.com/questions/68459401/soundcloud-api-authentication-always-throws-401-unauthorized

[bretonium]

looks like it. I am constantly getting mopidy[7898]: ERROR [SoundCloudBackend-6] mopidy_soundcloud.soundcloud Invalid "auth_token" used for SoundCloud authentication!

[bretonium]

According to soundcloud/api#151, this will not be fixed. Soundcloud api can now be used only together with a service that will issue a token. The tokens also need to be refreshed, since they are now time-limited.

Can mopidy provide an API endpoint that the clients could use to fetch the tokens? Is there anything similar in mopidy already?

[kingosticks]

Yes, Mopidy-Spotify works this way (utilising https://github.com/adamcik/oauthclientbridge hosted at auth.mopidy.com). We should be able to re-use that here also. Although I am not sure who controls Mopidy's soundcloud app and if they would need to update anything in the account settings, maybe it is @adamcik ?

Although they (Soundcloud) maybe need to sort out their CORS issues also (from https://github.com/soundcloud/api/issues/151).

[adamcik]

I don't think I have access to the oauth things for this account. However, I did play with using the bridge for SoundCloud and it worked at the time.

So if we can recover the old account or get new credentials we should be good to go.

Perhaps check with @tkem if he has access?

But, ideally we should have this attached to a non-personal account like we've done for Spotify if at all possible.

[tkem]

@adamcik Well, sorry, no access.

[bretonium]

The current client_id was added in 2013 by @dz0ny

[kingosticks]

As far as I know there's still no way to get new soundcloud credentials (facepalm) but if the authorization flow worked previously as is, maybe we don't need to hunt down the owner (although it'd be nice to sort that too).

[adamcik]

That was with the old flow that got / is getting removed, so won't work :( Checked https://soundcloud.com/you/apps and I don't have anything setup myself.

[kingosticks]

@dz0ny has access and can provide he secret (is that all we need @adamcik) but everything else is locked out and can't be changed. This includes the callback URL but hopefully that isn't a show-stopper for us.

[adamcik]

That should be workable with the id + secret

[adamcik]

Thanks @kingosticks for getting the secret to me, I tried running this in my local test instance of the oauth bridge and I need to do one minor code change and it should all just work.

Only ugly bit we'll have to do is to update https://mopidy.com/soundcloud_callback with some JS and a fallback link that sends people to https://auth.mopidy.com/soundcloud/callback with the same parameters and then we are back in business.

So what we need to progress is:

• [ ] List of the scopes we should ask for
• [ ] Setting up the oauthclientbridge properly
• [ ] Update of soundcloud_callback to redirect if the "right" oauth param is there so we can handle old and new cases
• [ ] Update of the plugin to start using the new flow with refresh tokens
• [ ] Update of the extension page on mopidy.com to fetch the new token types

What did I miss?

[adamcik]

And by should just work I mean that I did manage to get a working token in my locally modified test setup.

[chris-aeviator]

is there a way where users can setup their own bridge in order to get soundcloud running again?

[adamcik]

Sorry for the delay, I haven't had much chance to work on my part of this.

@chris-aeviator no not really, my code is on github, but you need credentials from soundcloud that they've stopped handing out years ago :(

[chris-aeviator]

Ok - thanks for clarification - I can see that I can't register a new API account...bummer that soundcloud forces us to use their crappy player

[adamcik]

https://auth.mopidy.com/soundcloud is now up and running - since we can't change the redirect it sends us to https://mopidy.com/soundcloud_callback/?code=REDACTED&state=REDACTED and not https://auth.mopidy.com/soundcloud/callback as it's supposed to. I.e. next step is to add JS to /soundcloud_callback to detect that we are in the wrong place and send people to the right domain so the rest of the flow can complete.

With the bridge up and running none of the remaining work should be blocked on me, other than the one minor code change to the bridge that I only have in my dev setup. If we need to specify a default scope just let me know, and I'll update the config. But at least other contributors can pickup the other items I mentioned further up.

[adamcik]

The old case sends you to https://mopidy.com/soundcloud_callback/?code=REDACTED&state=REDACTED#access_token=REDACTED&scope=non-expiring and fails with eg https://mopidy.com/soundcloud_callback/?error_code=access_denied&error_description=user%20cancelled%20request&state=REDACTED

So for successful requests we can check if the hash is set and decide if we should redirect or not, but for errors I think our only option is to encode something in our state parameter :/

[adamcik]

Testing with changing the button href in https://mopidy.com/ext/soundcloud/ to point at the bridge and going through ends up failing with INVALID_STATE: Authentication page sent wrong state to pop-up. since we get the state that the bridge and soundcloud share, not the mopidy website and the bridge.

So perhaps we can just blindly assume that we should redirect this case to the bridge, and worst case it also fails the state check and no harm done, and in most cases we end up with a page that can handle the old and new cases.

[nimbling]

EDIT: NO IT DID NOT MY BAD, SORRY :(

Hopefully this helps: I had the mopidy_soundcloud.soundcloud Invalid "auth_token" used for SoundCloud authentication! line in my Mopidy log, but that disappeared after I authorized myself at https://developers.soundcloud.com/docs/api/explorer/open-api

[simaovienna]

As soundcloud "...will no longer be processing API application requests at this time...":

https://soundcloud.com/you/apps/new

is there any chance/workaround to get the integration to work? @nimbling how did you authorize?

thanks

[nimbling]

I didn't, please see my correction. I did just log in using that link here: https://developers.soundcloud.com/docs/api/explorer/open-api - but it did not help as much as I thought it did.

[zlocate]

After removing the client_id from the request url in the API, everything worked.

[Nick1296]

After removing the client_id from the request url in the API, everything worked.

Same here! I just removed the client_id in steamble_url and _get

[kingosticks]

And are they still working? According to the SC docs those tokens will expire after an hour. If that is true then you'd have to go and re-auth Mopidy with SC to get another token, and so on.

[Nick1296]

Tried loading a random song right now from soundcloud, it seems to be working. I'll keep trying here and there to see when the token expires

[kingosticks]

If people wouldn't mind giving https://github.com/mopidy/mopidy-soundcloud/pull/133 some test time that'd be handy. I don't really use it.

[photoangell]

Pulled your fix and it works great for me. I did get a one time log entry when first playing a song...

INFO     2022-05-20 13:47:29,697 [7:SoundCloudBackend-18] mopidy_soundcloud.soundcloud
  Streaming of public song using public client id failed, trying with standard app client id..

but it doesn't seem to affect playback and I haven't seen the message since. my config is

[soundcloud]
enabled = true
auth_token = xxxxxxxxxxxxxxxxxxxxxxxxxx
explore_songs = 100

[kingosticks]

Yep, that message is normal. And it shows you are actually using the (standard) logged in client ID, which is the one we have hopefully now fixed.

[blacklight]

How did you folks manage to get an auth_token? After authenticating to https://mopidy.com/ext/soundcloud/ I don't get any usable token (nor errors on the console). I have tried with the oauth_token from an active web session too, but that didn't work either.

[Homer-Sim]

Sorry for my question, how do I pull a fix?

[kingosticks]

Search online how to install from git using pip. Failing that I'll cut a release tonight. I have my doubts about this hack and we probably still need to do the real fix that @adamcik laid the groundwork for.

@BlackLight it works just the same as our Spotify auth. Click the button, do what the Soundcloud login popup says, that popup will then close and an auth token have appeared in the box for you to put in your config. I think the page says all this, not sure what could be going wrong as I just checked and it's working fine.

[blacklight]

@BlackLight it works just the same as our Spotify auth

Thanks, I've eventually managed to get it to work in a clean session of Chromium with no extensions. I guess that the paranoid settings and extensions that I've got in my browser were blocking some CORS request or the target SoundCloud domain, but oddly I didn't see any errors in the console nor on the XHR requests that I usually see in these cases.

[Homer-Sim]

Search online how to install from git using pip. Failing that I'll cut a release tonight. I have my doubts about this hack and we probably still need to do the real fix that @adamcik laid the groundwork for.

I did a apt remove of existing installation and a pip3 install "git+https://github.com/mopidy/mopidy-soundcloud.git" which tells me about a successful installation service restarted but no soundcloud options in webgui?

Did I something wrong?

[kingosticks]

As you're using the service you need to do sudo pip3 blah blah for a system-wide installation else the system service user cannot see the installation. You can just redo your command with the added prefix. I'm sorry, I should have mentioned that as it's not obvious and it's particular to programs running as other users (e.g. services).

[Homer-Sim]

Works like a charm

[johnhamelink]

I'm excited to get Soundcloud working with my mopidy setup!

I've tried creating a new auth token for myself, but after I complete login and the popup closes, I get an alert box with the following:

INVALID_STATE: Authentication page sent wrong state to pop-up.

Edit: I think I might have been rate-limited, as I can now produce an auth_token, but when I boot mopidy I still get the old Invalid "auth_token" used for SoundCloud authentication!. I'm installing mopidy-soundcloud with python3 -m pip install "git+https://github.com/mopidy/mopidy-soundcloud.git"

Edit2: I tried again some time later, and lo-and-behold pip install from git works, and I have a working soundcloud!

[dominikdeka]

@kingosticks I have installed Mopidy-Spotify from master: sudo python3 -m pip install "git+https://github.com/mopidy/mopidy-soundcloud.git" I have restarted Mopidy service for many times, I have generated a new auth_token... and I am still getting [SoundCloudBackend-6] mopidy_soundcloud.soundcloud Invalid "auth_token" used for SoundCloud authentication!

Am I missing something?

[dominikdeka]

I have installed it again this time with command: sudo python3 -m pip install "git+https://github.com/mopidy/mopidy-soundcloud@master" and this finally replaced the lib with master version. Its great to see mopidy-soundcloud working again. Especially considering the fact that workaround for mopidy-spotify is still in proress. Thank you @kingosticks !

[kingosticks]

Sounds like we need a release.

[phijor]

Is there a chance we can get a new patch release that includes the API changes? I maintain the mopidy-soundcloud package in the AUR, and I would prefer not having to manually patch that package.

If there's any blockers that I can help resolve, please tell me :)

Thanks to everyone involved in making this extension!

[Goosegit11]

Hello all, sorry if I ask stupid questions, I am new to GNU/Linux and I want to listen to soundcloud with mopidy.

so, which token should I put into "auth_token"? 1-1111111-111111111, which is on readme.md of this repo, or 1-11111-11111111111-abcdvteewq323176dfdwhich is on the mopidy-soundcloud auth page

and did I understand correctly that this is the right way to install mopidy-soundcloud now: sudo python3 -m pip install "git+https://github.com/mopidy/mopidy-soundcloud@master"

offtopic question: how do I turn off mopidy.service and use mopidy as user? do I just systemctl disable mopidy or what?

[dominikdeka]

In order to obtain authenticate key go to https://mopidy.com/ext/soundcloud/ and utilize Authenticate mopidy with SoundCloud button.

and did I understand correctly that this is the right way to install mopidy-soundcloud now: sudo python3 -m pip install "git+https://github.com/mopidy/mopidy-soundcloud@master"

Yeap, it worked for me two days ago. (I am afraid new realese still not yet ready. :( )

offtopic question: how do I turn off mopidy.service and use mopidy as user? do I just systemctl disable mopidy or what?

Yes, but as a root sudo systemctl disable mopidy. Also make sure you stop the service, before launching mopidy as user: sudo systemctl stop mopidy

[Goosegit11]

Jesus Christ, seems like I got it working now :skull:

For those who haven't figured that shit out yet:

don't use mopidy-soundcloud from AUR, don't use just sudo pip install mopidy-soundcloud, it's not working.

Use this:

sudo python3 -m pip install "git+https://github.com/mopidy/mopidy-soundcloud@master"

thank you @dominikdeka @kingosticks and all contributors of mopidy-soundcloud

[kingosticks]

So... I never made that release?! Yikes. Sorry

[dreamlayers]

There is no need to run pip as root via sudo. The installation only needs to be accessible to the user running mopidy. I used sudo -u mopidy bash to become the mopidy user and then installed without sudo, with the install ending up under ~mopidy/.local/lib/.