Docile-Alligator / Infinity-For-Reddit

A Reddit client for Android
https://play.google.com/store/apps/details?id=ml.docilealligator.infinityforreddit
GNU Affero General Public License v3.0
4.15k stars 406 forks source link

Imgur API "random" failure #1417

Open sgtlaggy opened 1 year ago

sgtlaggy commented 1 year ago

Checklist

App version

5.4.1

Where did you get the app from

F-Droid

Android version

11, LineageOS (irrelevant)

Device model

No response

First occurred

No response

Steps to reproduce

  1. Wait for all Infinity users to collectively make 12500 imgur api calls
  2. Attempt to visit any non-direct imgur link, such as an album

Example post, link, markdown...

No response

Expected behaviour

No response

Current behaviour

Infinity's Imgur API calls receive the X-RateLimit-ClientLimit/X-RateLimit-ClientRemaining headers, limiting all users collectively to 12500 calls per day and if that is exceeded, as it seems it was last night, more than 5 times the app will not be able to load any more indirect imgur links for the rest of the month. From Imgur API docs:

Each application can allow approximately 1,250 uploads per day or approximately 12,500 requests per day. If the daily limit is hit five times in a month, then the app will be blocked for the rest of the month. The remaining credit limit will be shown with each requests response in the X-RateLimit-ClientRemaining HTTP header.

Interestingly, RedReader uses the same authorization header and Imgur API version but only receives the user ratelimit headers, not client headers.

I discovered this through inspection with mitmproxy. Sample request and response from each app provided below. I wasn't able to use the same URL as doing so resulted in a cache hit and returned the same response, all headers included and unchanged despite the different client ID.

Infinity
Request
GET https://api.imgur.com/3/album/Kq21p99 HTTP/2.0
authorization: Client-ID cc671794e0ab397
accept-encoding: gzip
user-agent: okhttp/4.9.1
content-length: 0
Response
HTTP/2.0 200
content-type: application/json
access-control-expose-headers: X-RateLimit-ClientLimit, X-RateLimit-ClientRemaining, X-RateLimit-UserLimit, X-RateLimit-UserRemaining, X-RateLimit-UserReset, Geo-City, Geo-Region, Geo-Metro-Code, Geo-Country-Code
x-ratelimit-clientlimit: 12500
x-ratelimit-clientremaining: 7265
x-ratelimit-userlimit: 500
x-ratelimit-userremaining: 499
x-ratelimit-userreset: 1679511884
etag: W/"a9c3fae6b0a2b2dfe4a9c622b36548c27a93210a"
cache-control: max-age=60, stale-while-revalidate=600, stale-if-error=86400, public
accept-ranges: bytes
date: Wed, 22 Mar 2023 18:04:45 GMT
age: 0
x-served-by: cache-iad-kjyo7100139-IAD, cache-bur-kbur8200100-BUR
x-cache: MISS, MISS
x-cache-hits: 0, 0
x-timer: S1679508285.911398,VS0,VE128
vary: Accept-Encoding
access-control-allow-methods: GET, PUT, POST, PATCH, DELETE, OPTIONS
server: cat factory 1.0
access-control-allow-origin: *
access-control-allow-credentials: true
x-frame-options: DENY
access-control-allow-headers: Authorization, Content-Type, Accept, X-Mashape-Authorization, IMGURPLATFORM, IMGURUIDJAFO, sessionCount, IMGURMWBETA, IMGURMWBETAOPTIN, X-expSNKPK127, X-Imgur-Defender-Bypass
content-length: 2303

{"data":{...},"success":true,"status":200}
RedReader
Request
GET https://api.imgur.com/3/album/gjP9hfm HTTP/2.0
user-agent: org.quantumbadger.redreader/1.20
cache-control: no-cache
authorization: Client-ID c3713d9e7674477
accept-encoding: gzip
content-length: 0
Response
HTTP/2.0 200
content-type: application/json
access-control-expose-headers: X-RateLimit-ClientLimit, X-RateLimit-ClientRemaining, X-RateLimit-UserLimit, X-RateLimit-UserRemaining, X-RateLimit-UserReset, Geo-City, Geo-Region, Geo-Metro-Code, Geo-Country-Code
x-ratelimit-userlimit: 500
x-ratelimit-userremaining: 499
x-ratelimit-userreset: 1679511884
etag: W/"f9bc6c363862005723e3d680e25dfb125e5244e7"
cache-control: max-age=60, stale-while-revalidate=600, stale-if-error=86400, public
accept-ranges: bytes
date: Wed, 22 Mar 2023 18:05:28 GMT
age: 45
x-served-by: cache-iad-kiad7000051-IAD, cache-bur-kbur8200142-BUR
x-cache: HIT, MISS
x-cache-hits: 1, 0
x-timer: S1679508329.715355,VS0,VE65
vary: Accept-Encoding
access-control-allow-methods: GET, PUT, POST, PATCH, DELETE, OPTIONS
server: cat factory 1.0
access-control-allow-origin: *
access-control-allow-credentials: true
x-frame-options: DENY
access-control-allow-headers: Authorization, Content-Type, Accept, X-Mashape-Authorization, IMGURPLATFORM, IMGURUIDJAFO, sessionCount, IMGURMWBETA, IMGURMWBETAOPTIN, X-expSNKPK127, X-Imgur-Defender-Bypass
content-length: 12719

{"data":{...},"success":true,"status":200}
### Logs _No response_
sgtlaggy commented 1 year ago

For reference, I saw https://github.com/Docile-Alligator/Infinity-For-Reddit/issues/1049#issuecomment-1296198859 and https://github.com/Docile-Alligator/Infinity-For-Reddit/issues/1375#issuecomment-1441776653 but figured this could use a more thorough investigation. I'm not sure if the latter implied this specific issue (and cause) was known and someone was talking with Imgur to figure out a solution.