search

benfoxall / lastfm-to-csv

Web based tool for exporting scrobbles
http://benjaminbenben.com/lastfm-to-csv/
MIT License
493 stars 85 forks source link

Last.fm to csv freezes after 600 fetched pages #9

Open hans-juergen opened 9 years ago

hans-juergen commented 9 years ago

I have tried to download my listening history of 1,164 pages several times now in different browsers on several machines, and it looks like the site has a memory leak, as I only have one browser tab open, and the task manager shows 750 MB RAM usage only for Last.fm to csv after about 600 pages. Sometimes it gets to about 800 pages, but is very slow then until I have to close the browser. Or is there some kind of timeout now maybe, either in the API or in your tool?

My Last.fm username is hjbardenhagen, by the way, and I didn't have any problems fetching my listening history about a year ago which was about 15% smaller then.

benfoxall commented 9 years ago

@hans-juergen ah, sorry - I've been updating it to work differently (the new version stores responses so they don't have to be downloaded again).

I've rolled back the site to the old version; so hopefully it should work for you again. I'll make sure I do some proper testing of the new version when I work more on it.

Thanks so much for logging the issue, sorry that it wasn't working for you (totally my fault).

benfoxall commented 9 years ago

@hans-juergen out of interest - what are you using your data for? I'm giving a talk next week, and I'd love to have some examples of what people can do when they have access to their own data.

hans-juergen commented 9 years ago

Hi Ben, thanks for your quick reply, I'm trying again now in Firefox 40.0.3 on Windows 10.

Your tool is very useful, thank you very much for creating and releasing it. :-) The official data exporter on the old Last.fm site did not work for everybody, so it was my only chance to have my complete listening history in a nicely formatted CSV file which the online tool Ivy could use to create large Spotify playlists from it and so recreate Last.fm's library radio in the Spotify desktop client. I have written a short description how to do this in a pinned thread of the old Last.fm for Spotify support group:

http://www.last.fm/group/Last.fm+for+Spotify/forum/1249115/_/2235908

Furthermore this method is still or again of interest for other users, because the latest Spotify versions 1.0.12 and 1.0.13 have problems to play a Last.fm radio station continuously, they stop after one track now when you start them on the new Last.fm site with a play link. That is why I have created a new related thread yesterday in the new forum:

https://getsatisfaction.com/lastfm/topics/avoiding-current-spotify-problem-with-playing-only-one-track

OK, fetching my listening history has finished now, 23 errors, but almost all pages included, as it seems (227,837 scrobbles instead of 232,638). Thanks again for fixing it so fast. By the way, it would be nice if there were some details about the errors, so I could estimate if I should try it again on another time of day. In my experience these errors come up when the API is under heavy usage, perhaps due to the bank holidays in the UK today.