felhag / lastfm-stats-web

A small project to show some additional statistics for last.fm.
https://lastfmstats.com
249 stars 12 forks source link

Large Account Loading Issues #56

Closed gryphon-0 closed 1 month ago

gryphon-0 commented 3 months ago

Hello,

First off just wanted to say I think this is the best last.fm stat site. Because of that, I like to load other peoples accounts into it and see theirs stats as well. There's been one account with the weirdest scrobbling behavior but they have over 3 million scrobbles. Upon loading the account, it got stuck occasionally but I would use the CSV export feature to create checkpoints. Unfortunately, around 5 years of account history and 700,000 scrobbles, the program no longer works. For a time it would at least grab a single additional scrobble, but now it makes zero progress.

I realize this may be a fringe use with these superuser accounts, but if there's any workaround. I've tried over the course of several days, using CSV, JSON, and indexedDB but I can't seem to get any more progress. I've also tried filtering the timespan.

If there's some sort of imposed limit then no worries. The CSV file is currently approximately 53 MB and contains 704,092 scrobbles from Sep 29, 2008 to Dec 31, 2013. Too large for the github 25MB limit so hopefully this link works: https://drive.google.com/file/d/1NZwQNRnv3ocTpNbFU3Qv3G1adn95L7RG/view?usp=sharing

Thank you!

felhag commented 3 months ago

It looks like it's some kind of issue with the lastfm api. For large accounts the api tends to get a little wonky after downloading some pages. You can see this by opening your developer tools and look at the network tab, it just suddenly gives 500 responses with the message: Operation failed - Most likely the backend service failed. Please try again.. I even tried downscaling the page size (from 1000 to 200) if a page download failed a few times but even smaller pages don't always seem to work.