Persistent HTTP 500 Error on Playlist Loading

[salvioma]

Hi,

Thank you for addressing the previous issue, but I’m still encountering a problem. The web interface is stuck on "http://localhost:8000/#playlists" with an HTTP 500 error. The page keeps loading without displaying playlists.

Error Details:

• Issue: HTTP 500 response when trying to load playlists.
• Impact: Unable to access the playlists.
• Reproduction: Occurs consistently when accessing the playlists page.

Could you please help resolve this issue? Thanks!

[pavelkomarov]

Localhost? So you cloned and ran python3 -m http.server yourself? It's also hosted at exportify.net. Does it also not work there?

If there's a 500 response when trying to load your playlists, then maybe Spotify can't fetch all your playlists for some reason. I'm now loading all user playlists on one screen, so I need them all to be findable and returnable by Spotify--which they should be, but the Laws of Large Systems could goof that up. How many playlists do you have total? And can you see them all through Spotify's website and in their app?

I've never heard of a user having this problem before. It may have to do with your unique playlists, like one of them getting corrupted server-side or something, making this one difficult to reproduce and address. Honestly, if Spotify's server doesn't want to cooperate, there's not much my web app can do to solve the problem.

[salvioma]

Hello, Pavelkomarov,

Until recently, I had always used exportify.net for convenience. However, as the issue persisted, I decided to download the code and run it locally in an attempt to resolve it. After making some adjustments, such as replacing the deprecated sklearn package, the error still occurs.

Given that I have only 5-10 playlists, I doubt the issue is related to quantity. It may be due to an invalid token from the Spotify API, as I’ve encountered similar errors with other services relying on this API.

I would greatly appreciate your help in investigating this, and if necessary, we could also consider contacting Spotify.

Thank you for providing this valuable tool. It has been incredibly helpful, particularly for organizing playlists based on the detailed genre tags, which allow me to curate lists with precision and consistency.

I look forward to a resolution. Thank you again for your time and attention.

[pavelkomarov]

Exportify.net is always up-to-date with the most recently-pushed repo code, because I've set up continuous deployment. I would expect things to work exactly the same there as at localhost. I've fixed the sklearn thing, which was only ever relevant to the python analysis side of this work, so you should be able to use the latest from the master branch.

A 500 response indicates there is an issue on Spotify's end (the server), and it's not a very specific error code, which means there may be nothing I can do from client-side. Does the issue happen when you try to log in or when you try to load playlists? Those things happen one after the other, so it's difficult to tell without looking at the actual traffic. What was the last request sent to the server, to which it returns HTTP 500?

Can you can log in on their website and see your playlists there? It's odd you're the only one I've heard from with this problem. You said you've encountered similar errors on other Web-API-based services, so that makes me wonder whether there is something odd about your specific account. If you find other examples of client-side apps where you can or cannot access your data, I'd be interested to know what they might be doing differently. Otherwise you may just need to try to get Spotify's help.

[salvioma]

I can access my playlists without any issues directly from https://open.spotify.com/playlist/, so it seems the problem is not related to my Spotify account, but rather how Exportify is interacting with the Spotify API in my case.

When I have the chance, I'll check the developer console in my browser to capture the last request before the HTTP 500 error. I’ll get back to you with more information on the failed request (endpoint and response) for further investigation.

Thank you for your assistance.

[pavelkomarov]

Do you have the same trouble if you use the version hosted at watsonbox.github.io/exportify? As far as I know he's doing authentication the same way, and since you have <20 playlists, he should be fetching your list of playlists the same way too. It would be illuminating to try, because if it also doesn't work, the issue is probably beyond us on the client side, and conversely if it does work we can ask what the two versions are doing differently.

[salvioma]

Subject: Issue with /v1/me/tracks Endpoint Due to Large Liked Songs Playlist

Hi Pavel, Thanks for your help with previous issues. I’ve been investigating further into the problem I’m facing, and wanted to share what I’ve discovered so far in hopes that you might offer some additional guidance.

It seems the HTTP 502 error I encounter when trying to load data from the /v1/me/tracks endpoint isn't related to the API functionality or the Access Token, as I can access other endpoints like /v1/me/playlists and /v1/me/albums without any issues. After deeper analysis, I believe the problem lies with my Liked Songs playlist.

I currently have over 57,000 songs saved in Liked Songs, and it seems the server struggles to process this volume, which likely results in the 502 error. Interestingly, when I had around 50,000 songs a month ago, Exportify worked just fine. This leads me to wonder if a recent change or update could be affecting the handling of large song lists.

Luckily, I’ve been able to use a secondary account with far fewer saved songs to successfully export the playlists I need, but I’d prefer to keep using my primary account.

My main questions are:

• Is there any way to modify Exportify to skip the Liked Songs playlist during export, so I can use my main account without hitting this issue?
• Do you think this problem could be tied to a recent update in the Spotify API? I was able to export my data when I had 50,000 songs, but now with 57,000, I’m encountering the error.

Best regards,

[pavelkomarov]

You indicated the 502 error occurs when building out your list of playlists. At that stage, I fetch a single song's metadata from your liked songs, because the return message contains the total number of liked songs. I can then use that info to create an identical data structure to that which comes back for playlists, which allows me to effectively treat liked songs as a playlist (so the rest of the code just works without modification).

The web API definitely shouldn't fail just because that number is 57k as opposed to 50k. Report this bug to Spotify on their forums. (You should include for them a minimal query that specifically causes the error. No promises they'll make a fix. I've had stuff fixed, but also had stuff languish there.)

You can see where I form the list of playlists in the code, exportify.js. I query the liked songs endpoint near the top, which sounds like it must be the query that precipitates a server failure. You can form that query for yourself and send it off using curl to replicate the failure in isolation (as shown in the web API. Don't worry; I can coach more of you need, but I'm currently still in bed just on my phone).

Note you'll need your access token to do that. Easiest way to get an access token is to print one from the exportify code. You were already hosting and running it yourself, so it's as easy as adding one line, a console.log. You can then read it off from the console (cmd+opt+C in Safari, for example, I think opens it). Each access token is good for only an hour.

Thanks for pouring so much thought in to solving this mystery.

[pavelkomarov]

And to answer your two ending questions:

I don't want to make skipping liked songs an option, because in order to not fail in your case I'd have to default to not displaying them, and it was the most-requested feature.

I just added this feature not that long ago, so it could have been my changes that resulted in the extra query that doesn't work for you now. I don't think Spotify has made serious changes to their web api in that same period, but I don't actually know. My guess is it always didn't work over some number, that the server can't find the metadata for some individual song among that 57k or is somehow trying to count things and timing out.