Export All problems

[shape55]

In Firefox, if I click "Export All" on the first page, I get a pop-up window that shows the number "400" w/ nothing else (I have 470 playlists). If I hit OK, it pops up again a couple times, and at the bottom of the page is the usual API Rate Limiting error.

[pavelkomarov]

So that number 400 appears in the alert box (line 31 of exportify.js). It's an HTTP code. I updated the error message to be more than just the number so you the user know that too.

The problem, at least where I was able to replicate it, turned out to be that artists and features can be null, and the code was then adding those nulls in to a query's comma-separated list, which resulted in double comma, which the Spotify server considers mangled.

I just pushed an update to fix this. Let me know if it works. That commit also addresses #1, the rate-limiting error, by awaiting each call to csvData in ZipExporter's export.

[shape55]

Ok, I didn't know if the 400 error code & rate-limiting error were related or not. I tried "Export All" again and I do not get any errors. Instead, the circle inside the Export All button gets stuck rotating indefinitely. The individual Export buttons work fine though to download 1 playlist at a time.

[pavelkomarov]

Shouldn't be indefinite. You just have a ton of playlists. If you look under the Network tab in developer tools (ctrl+shift+E), then you should see new unique requests that whole time. Is that not the case? Also, check for me that when it finishes you have all hundreds in the zip, not just the 20 from the first page.

[shape55]

Ok, well I tried it again in the mean time, so I'll check the Network tab now. The first time I tried I let it run for over an hour so I didn't think it was going to download anything.

At the very bottom-left next to the clock icon, for Number of requests it says "No requests", yet the Export All icon has been spinning for about 20 min.

[pavelkomarov]

Hmm. Yeah, something is wrong. I make the button spin when I start exporting (line 167 in the .js) and set it to not spin any longer when it finishes and reaches line 174. The script is dying in between. Open the browser terminal (ctrl+shift+J), and see what the error message is. Let me know how to replicate the problem if possible.

You weren't seeing any requests in the Network tab because by then no new requests were being generated. Open that tab before you click Export All, click it, observe the behavior, and report back.

Also, just tried it, and Export All only grabs the ones on that page, which I thought might be the case. I should probably update it to grab all 400+ in your case. That's more intuitive behavior. I opened issue #9 for this.

[shape55]

This was the first error I got. It popped up after reloading the page while the button was stuck on "Exporting": TypeError: NetworkError when attempting to fetch resource.

This is the only message that appeared in the Console thus far after hitting the Export All button 5 minutes ago: TypeError: features[i] is undefined exportify.js:260:29

[pavelkomarov]

Would you be willing to change your Spotify password to something innocuous and email me the account login for debugging purposes? Ideally I get some print statements up in this piece. There's probably some corner case that will take us forever to diagnose otherwise. You can try really getting in there to diagnose it yourself if you want.

That error on line 260 doesn't make any sense to me, because features[i] should either be filled or should be an array of undefineds. (Line 240) Try printing all of features right around there. Maybe the whole thing is null.

Why are you getting a NetworkError while I'm not with any other account I've tested this on? It's possible that came from a different tab. The terminal catches stuff from everything that's open.

[pavelkomarov]

My guess is this is failing on a particular playlist with some odd characteristic. You say the individual Export buttons work, but have you verified it works for all 20 on the page? I bet you get the same error for one of them, and you just didn't happen to click that one when you checked the basic Export functionality.

[shape55]

After my last comment I tried the the suggestion in your 2nd paragraph w/ the Network tab. I opened the tab before I clicked Export All and this is what I got. Note that before I hit the "Get Started" button there were 15 requests.

325 requests 15.65 MB / 1.97 MB transferred Finish: 5.40 min DOMContentLoaded: 1.73 s load: 1.76 s

When I tried it again I got this:

286 requests 15.65 MB / 1.95 MB transferred Finish: 4.12 min DOMContentLoaded: 1.35 s load: 1.36 s

As far as the Network error in the Console, I have no idea. I'm not having any issues with pages loading. My connection seems fine otherwise.

If this doesn't give you the info you need, then sure, I can change my Spotify pw if that'll help you figure it out quicker.

[pavelkomarov]

My email is pvlkmrv@gmail.com. I'll try to get this turned around quickly and give you back full control. But I appreciate it. If we can make it work with your insane account, then hopefully it works for everyone.

OR: If my hunch is right and this is caused by a particular one or few playlists, we can get away with making those public (if they're not already), me following them, then using my account to try to export them.

Try flipping over to the second page to see if Export All works there. Repeat for the third, etc. On any (20+ of them for you) it doesn't work, go through item by item looking for the troublemakers, and give me links to them so I can follow.

[shape55]

Actually, you are correct there b/c there is one and only one playlist that I have not been able to export for some reason when I did previous backups. It's on the 2nd to last page (441-460 of 470 playlists). I've tried using 4 different Exportify forks at this point on both Firefox & Chrome. It can't be b/c there's too many tracks since there is another playlist that has even more tracks that downloads fine. I'm wondering if it's b/c of a special character. It's the only playlist that has music notes in the title, but plenty of other playlists have different special characters.

Here is the link if you want to see if it's reproducible on your end. Note, this is a playlist I subscribed to, not one that I created: spotify:playlist:53eljwwaQQhi93Zbet44si

I'll try temporarily unsubscribing from it and then test the Export All button again.

[pavelkomarov]

Again, Export All right now only exports the 20 shown on the current page, so for any page it doesn't work, there's at least one of the 20 that's causing some issue.

Yep! I'm getting a problem for that one. One of the song.tracks is null. Goofy. Has nothing to do with the name having special characters at first glance.

Let me know of any others you find. I'm going to close this issue again now, because it's not really an Export All problem; it's single playlists. Open a new issue for them, and I'll get around to investigating.

[shape55]

I posted that before I saw your followup message.

In regards to exporting the 20 shown on the current page, you mean with my playlists specifically? You were able to determine that with the Network requests?

Ok, glad you were able to reproduce it. I unfollowed that glitchy playlist and reloaded Exportify, but the Network tab still stops loading them at the 325th request. And that's when I get the "undefined exportify.js:260:29" error.

I recently backed up all the other playlists though using another fork of Exportify and they all downloaded w/ no issues. Is there anything at the 325th request that would tell you what might be causing the error?

[pavelkomarov]

If you click "Export All", and it hangs, then the problem is one of the 20 playlists displayed directly below, not this one at the very back.

I've been examining that one playlist, and there's a song called Walk Alone, added the 10th of December 2018. When it's fetched the JSON comes back mostly empty for some reason. If I add the song to a different playlist, everything works fine; fetching it gives me back a more complete object. I have no idea why this is, but it's definitely Spotify's bug. I'll keep digging in to it, but I think the solution is just to work around those cases. I'll open an issue to add some failsafes to the code so nothing crashes.