LOGIN IS BROKEN (fixed)

[alexweinstein]

Cannot login to Soundcloud via soundcloud.com or API connect flow. Error message: "Something unexpected happened. Please try again." Same for multiple accounts.

Started this morning. Is anyone else experiencing this?

UPDATE: Can now login again at 11:58 AM, Thursday, May 26, 2022 (EDT). What happened?

[mgoodfellow]

Hi, to add to this, they have added rate limiting to new endpoints and knocked everything offline:

"{\"errors\":[{\"meta\":{\"rate_limit\":{\"bucket\":\"by-ip\",\"max_nr_of_requests\":30,\"time_window\":\"PT1H\",\"name\":\"client-credentials-exchange\"},\"remaining_requests\":0,\"reset_time\":\"2022/05/26 16:31:52 +0000\"}}]}", time taken: 00:00:00.0358613

[mgoodfellow]

Ah, so about 15:30 UTC, all their token endpoints returned 500 internal service error, so we busted our caches while refreshing tokens and now have bounced off rate limits

[stephan-blum]

According to the corresponding email thread, it looks like the issue could be resolved on your side.

[mgoodfellow]

Hi @stephan-blum,

Thanks for the reply.

I think the issue is a bit deeper than that here with this particular use case.

As a general request - it would be great if we could get the documented rate limits for various endpoints/actions so we can configure circuit breakers correctly to not overload SoundCloud API and get locked out when an issue does occur. I'm aware of a global rate limit at SoundCloud: https://github.com/soundcloud/api/issues/189 and this is what we currently use with our circuit breakers.

However, in this particular case, it appears the rate limit is for /oauth/token, and specifically for the client-credentials-exchange. It was my understanding that we are limited to being granted 50 tokens in 24 hours (https://github.com/soundcloud/api/releases/tag/2021-09-07). This as a rate limit makes sense, and the rate limit is issued to a clientId, not an IP. However, an aggressive edge rate limit of 30 requests in 1 hour for a specific IP means we wouldn't have been able to protect against hitting this rate limit last night because the limits are far to low to allow for effective circuit breaking

What happened for us yesterday was that we tried to refresh a server token (client-credentials-exchange), and got back a 500. At this point, we have to cache bust our access token and refresh token as we don't know if the refresh token is valid anymore, and we need to go back to a consistent state. We then retry this token grant process until we can get a token, ignoring these server errors as the token hasn't actually been granted. As an example, even with retries set to 10s back off, we would still run out of available hourly API requests for this specific endpoint within a few minutes. For us the API was returning 500, which indicates a server side issue, and we should retry. Unfortunately this retry was counted towards our rate limits, and eventually we got locked out for an hour.

Does this new rate limit replace the 50 server side tokens per hour (https://github.com/soundcloud/api/releases/tag/2021-09-07)? Or is in addition to?

Are there any rate limits on other endpoints other than the global one I mentioned earlier? It would be great to find this out so we can correct our circuit breaker settings going forward if they aren't right.

I would also request that this particular rate limit is lifted to something which we can effectively have a back off or circuit breaker approach with. 10 requests in a minute would be workable for example, but ideally higher than that for cases of failure. Or I guess if server side failures didn't count towards the consumption client rate limits that could be workable as well?

Thank you in advance for any input you can provide!

[dpreussler]

@alexweinstein FYI the rate limit for authentication is officially documented here: https://developers.soundcloud.com/docs/api/guide#authentication

Please be aware there is a rate limiting on amount of token you can request through the Client Credentials Flow: 50 tokens in 12h per app, and 30 tokens in 1h per IP address. In order to not hit the limit we highly recommend reusing one token between instances of your service and implementing the Refresh Token flow to renew tokens.

any chance you can move to authentication code flow instead?

[mgoodfellow]

Hi @dpreussler

Please the issue that occurred above. Your /oauth/token endpoint was returning 500s and not issuing tokens. Unfortunately, this consumed the rate limit of 30 "tokens" per IP.

Thanks!

[alexweinstein]

Closing. @mgoodfellow's comment now has a dedicated issue: https://github.com/soundcloud/api/issues/223