search

subject-f / cubarimoe

GNU Affero General Public License v3.0
60 stars 17 forks source link

Proxy for Reddit galleries. #5

Closed Fyren closed 2 years ago

Fyren commented 2 years ago

This uses the Reddit API. The docs say OAuth2 is required, but in reality, it doesn't seem to be for read-only stuff. I don't know if actual usage on Cubari will be any different, but testing on my local instance works (and people say they've been using the Reddit API without authentication).

I could instead scrape the HTML for the gallery, but that seems to expose a little less info about the post but also contains comment data.

It accepts two kinds of URLs: https://www.reddit.com/gallery/q0o2in https://www.reddit.com/r/manga/comments/q1wf0n/yugamis_request_yugamikun_ni_wa_tomodachi_ga_inai/

Let me know if you'd like any changes or if I should be doing something more to test than throwing a few Reddit URLs at it.

funkyhippo commented 2 years ago

Awesome work, thanks for contributing! I'll take a closer look and test when I have some time tomorrow, but have you verified the reddit API rate limits? I recall that there is a rather strict limit (regardless of the authentication requirement), so this could be a scaling issue if reddit constantly returns 429s to our proxy.

Fyren commented 2 years ago

I looked into their API rules/terms some more.

They don't say what the rate limit is for non-authenticated uses. For OAuth users, they indicate rate limits in the headers of the response. Using my personal Reddit account and OAuth, the limits were 600 requests per constantly-revolving 600s window.

The docs also say they care about the user agent in some way and not to spoof a browser. They want you to use something like "SomeName/0.1 by u/redditname".

They docs say API users have to agree to their terms and register using a Google form here: https://docs.google.com/forms/d/e/1FAIpQLSezNdDNK1-P8mspSbmtC2r86Ee9ZRbC66u929cG2GX0T9UMyw/viewform The terms look harmless, to me, but the form wants an e-mail, phone number, description of what you're using the API for, and also about how/if you're "monetizing" your API use. Without using the form, I was still able to make OAuth requests.

Fyren commented 2 years ago

Let me know if you want further changes regarding the error handling.

funkyhippo commented 2 years ago

Merged and deployed. Thanks!

Example