Possible solutions for bloodcat

[fluddokt]

I guess an easy solution would be to just open the page in the browser. But -the user would have to choose to download to the Import directory so opsu could import it -loses the internal download progress. Though it could continuously check the current filesize if they saved it the import directory for progress but still need the find the end filesize somehow. -still provides the benefit of avoiding duplicate beatmaps at least.

Another way would be to somehow integrate the reCAPTCHA. Not really how sure how possible this is. (Can at least get the page in the error stream)

Might be able to do this with osu website also. (Open in browser or integrate login somehow) https:// osu.ppy.sh/d/%d[n]

[tpenguinltg]

I don't think it would be very easy to integrate reCAPTCHA without having to import a Web renderer (including a JS engine), and I think that would be outside the scope of this project.

I just tested another possible solution: passing in a session cookie. When you solve the captcha, it stores a session cookie so that you don't have to solve it again. It seems it doesn't validate the originating IP address, so you can successfully download from any IP address. The only thing I'm not sure about is how long the session lasts for. The cookie expires in the browser after 3 days, but that doesn't necessarily mean that the session does, especially if requests are made before the 3 days ends. More testing will need to be done, but assuming the session duration isn't fixed, then the solution would be to solve the captcha the first time, take the session cookie, and send it along with the request to the beatmap. For safety, there would be some periodic "keep-alive" process that would download a map with the session cookie.

As for where to put the cookie, it can either be hard-coded or put in the config file. Hard-coded would mean that updates are easier, but updating the cookie requires a new release. Putting it in the config file would mean that updates don't require a new release, but updating it would be more difficult.

[fluddokt]

Seems interesting but I'm not sure what to do with that cookie, since it probably expires eventually anyways unless the user can somehow get their own cookie into opsu themselves.

edit: Yeah the problem is that someones going to need to update it every time it expires and I think bloodcat will eventually catch onto that(AnyIP) too.

[tpenguinltg]

Actually, a cookie is really only a "key=value" pair, and expiration is only a thing in a browser. You can send the same "key=value" pair with your request long after the browser cookie expires and it will still be a valid cookie. Whether bloodcat will catch on is uncertain, but it is a valid concern.

[tpenguinltg]

As for osu! login, it's probably much simpler. It looks like it's just a POST request to https://osu.ppy.sh/forum/ucp.php?mode=login with fields username and password (and possibly also redirect=/, sid, and login=login), then you'd just keep the cookie around while the program is running. The username and password would be stored in the config file (I think osu! keeps it in plaintext anyway).

[fluddokt]

Huh I always thought the server would expire the session also. Still seems like a pain if it does expire. Yea osu login is totally possible cool.

[itdelatrisu]

The cookie approach seems interesting (didn't know reCAPTCHA worked like that), but problems I have with approach:

• I don't really want to write or maintain this solution for Bloodcat given that the Bloodcat API is probably not stable (and it's already a huge pain to maintain all the download server interfaces), or reCAPTCHA might change too.
• peppy is probably not cool with us hitting his servers for a game clone (at least programatically).

Integrating reCAPTCHA seems out of the question. Who knows, maybe Bloodcat will even get rid of captchas once their traffic dies down (which I imagine it will pretty soon, with everyone's downloaders now broken).

One thing I've considered in the past is using one API for searching and another for downloading (e.g. Bloodcat for searches, and MengSky for downloads). Presumably, there will be a lot more search servers online at any point than download servers; users can select a "search server" and "download sever" separately this way. I haven't done this yet due to lack of motivation/time.

Simplest solution seems to just be launching a web browser to download, and have users move the files into the Import/ folder (in this case, we might want to improve the UI to show people how to import the files without restarting the client).

@fluddokt: Not sure if you've seen me post this yet, but MengSky is still working pretty well (and it's much faster than Bloodcat for me). They just enforce Referer headers now (see 06370bd), and I haven't made a new release with this yet.

[fluddokt]

Yea I think opening it in the browser is the solution.

Oh >.> haven't actually tried that. Saw the issues and assumed they all didn't work. Might update my port to 0.15.1 or something.

[tpenguinltg]

Simplest solution seems to just be launching a web browser to download, and have users move the files into the Import/ folder (in this case, we might want to improve the UI to show people how to import the files without restarting the client).

Better yet, we can also have an "Import" button that shows a file picker dialog.

The cookie approach seems interesting (didn't know reCAPTCHA worked like that)

I think the session thing is specific to bloodcat, but I haven't done any research on it.

[tpenguinltg]

I was reminded today that I set up a cron job last week that downloads a beatmap every day with the same cookie to test the cookie solution and it looks like it's still getting a ZIP file 7 days after the session was started.

Would it be desirable to turn download servers into plugins, perhaps with some sort of ClassLoader? That way, if someone wanted to maintain a solution for Bloodcat, for example, it wouldn't have to be official nor supported. Any security risks taken on by using such a plugin would also be on the user.

[itdelatrisu]

Sorry for the delay; I added at least a partial implementation in ab39b3e. This just opens a browser, nothing fancy that was discussed above.

Adding an "Import..." button with a file picker seems okay, but I'm hesitant because it'd actually be slower for users to do that each time than to just download files into the Import/ directory or change that directory to their own Downloads/ folder.

@tpenguinltg: Plugins probably aren't happening since it'd be undoubtedly a dead feature. :P