search

tomhepz / Osu-Most-Played-Downloader

4 stars 2 forks source link

Working!! but how to fix osu server timeout #2

Open AustinKol opened 2 years ago

AustinKol commented 2 years ago

I'm getting the 429 osu timeout request and can't download more than a couple dozen maps at a time. Other than that, it's working like a charm!

bobokun commented 1 year ago

Hi @AustinKol , not the original developer but if you want to test my fork it should fix several issues.

AustinKol commented 1 year ago

Hello @bobokun, thank you for taking the time and creating the fork. Although you wrote some good code, unfortunately, I don't think simply forking it is enough to make things work. This is because there's a more fundamental issue.

Request CAN'T actually download the osz files with the map files inside, it simply just zips up the web pictures found on the beatmap download page. I still don't quite understand how it works but let's just say peppy wrote some genius code that hashes the user timestamp into the actual download link. Therefore, unless someone physically clicks the download link, it's impossible to download the beatmap.

This is why I went on a completely different approach. I ended up using OP's code to generate a list of beatmap ids in a text file then downloading them using chromedriver. It's not user friendly yet but I created a google drive folder with the code inside : https://drive.google.com/drive/folders/1ZmEpbYylGQpnH59VcraZprsay1L_hsJz?usp=sharing The readme will explain what each file does.

Please try it out and lmk if it works! So far, it worked out great for me, just a bit slow but you can adjust the sleep timings in beatmap_downloader.py. Just make sure it doesn't time out. I also addressed the issue where it can only get the beatmap ids of 100 beatmaps at a time just by adding a while true statement lol.

bobokun commented 1 year ago

Wow you're absolutely right. I should've tested more thoroughly, how silly for me to assume the files that were being downloaded are able to be imported. Thanks for sharing with me your code! I'll take a look and give it a shot 🙂

bobokun commented 1 year ago

@AustinKol your session cookie is hardcoded in get_beatmap_id.py You probably want to remove that reference and refresh your cookie.

Also I'm not seeing the readme file, but I should be able to figure out how it works nonetheless. I'll let you know if I have any questions!

AustinKol commented 1 year ago

@bobokun thank you for warning me about the session_cookie! I apologize for not having the readme in time. I was in the states until yesterday and had limited resources to write it. It is currently in the google drive folder.

I'm glad you're showing interest in my code! I was very desperate in coding this because I accidentally lost all my beatmaps during a windows reinstall. Please let me know if you have any questions.

bobokun commented 1 year ago

Thanks @AustinKol , I took inspiration of your code and implemented an all-in-one solution that is working now. :) I've updated the PR so it should work standalone. 👍

tomhepz commented 1 year ago

Thanks for the contribution, I'm very busy with studies at the minute. When I get time i will review and merge.

AustinKol commented 1 year ago

@tomhepz nppp. all for the osu community. @bobokun I tested your code and it works flawlessly it seems. damn that's quite genius what you did. My code had the inconvenience where chrome needed to be constantly opened but you fixed it.