🐛 Bug Report: Rate Limit error

SteffenBauer commented 2 months ago

Describe the bug

Redlib started to throw JSON parse errors since yesterday (#229), caused by 403 errors from reddit. Redlib was patched very quickly, but since then Rate Limit errors occur. This is already reported in issue #229 , but I think as this is a new issue it makes sense to open a new bug report.

Steps to reproduce the bug

Install latest redlib commit (https://github.com/redlib-org/redlib/commit/8ef45456d696b254b0378b75ff4db0345d6bc217)
Browse redlib for several minutes
Observed: After some minutes, redlib starts to fail with Rate Limit error.
Refreshing the page on browser does not change back, still throws rate limit error, even after waiting for some minutes.

What's the expected behavior?

Redlib continues to show reddit content

Additional context / screenshot

Couldn't send request to Reddit: Rate limit - try refreshing soon | /r/popular/hot.json?&raw_json=1&geo_filter=GLOBAL

Message in systemd journal log:

Sep 17 08:55:26 hufflepuff redlib[1763]:  ERROR redlib::utils > Error page rendered: Couldn't send request to Reddit: Rate limit - try refreshing soon

SteffenBauer commented 2 months ago

Additional information:

Redlib works again for some minutes when redlib service is restarted.
Refreshing the page in browser does not change the Rate Limit error, even after waiting for several minutes without any access to redlib in the mean time.
Local build from source, running as systemd user service on local home server. Tested with only me accessing redlib, light load (rate limit error occurs after browsing a few dozen reddit pages)

Tokarak commented 2 months ago

Exactly the same situation and experience.

Tokarak commented 2 months ago

Relevant code: https://github.com/redlib-org/redlib/blob/8ef45456d696b254b0378b75ff4db0345d6bc217/src/client.rs#L266-L270

pimlie commented 2 months ago

While testing that PR @perennialtech linked, I found that when I kept relatively quickly refreshing I sometimes was rate limited but could almost always recover by just refreshing the page. But when after N requests I stopped refreshing for X amount of time (like a minute or more) then I immediately got rate limited in the next refresh in a way that wasn't recoverable with a new token.

As restarting also seems to always work, it makes me think it might be a connection timeout / keep-alive related issue? Not sure if rust/redlib re-uses the same connection on successive requests to upstream?

sigaloid commented 2 months ago

Closing as duplicate of #229

redlib-org / redlib