SE API gives "throttling violation" when using FIRE

[user12986714]

After upgrading to Chromium 83.0.4103.116 (orig 80.0.3987.162), FIRE appears to stop working. The fire button on each report still loads, but nothing will happen when clicking that icon (there should be a dialog popping up showing the post and various feedback options). Rolling back to Chromium 80.0.3987.162 won't help.

Devtool indicates 1 error, GET https://api.stackexchange.com/2.2/sites?filter=!Fn4IB7S7Yq2UJF5Bh48LrjSpTc&pagesize=10000&key=[REDACTED]&_=[REDACTED] 400 (Bad Request) Stack trace indicates that this is from FIRE.

Directly visiting the identified url gives {"error_id":502,"error_message":"too many requests from this IP, more requests available in [WHATEVER] seconds","error_name":"throttle_violation"}

Platform linux, $CC=gcc 8.3.0, CPU architecture x86_64 Chromium 83.0.4103.116, Tampermonkey v4.10

[makyen]

Running (copy & pasting into the console) the JavaScript here should get you at least a bit past where you are now for FIRE.

There's not going to be a lot that FIRE can do about the throttling violation. It's also unlikely, but possible, that it's FIRE's fault. Throttling violations happen when

1. You, or your IP, makes too many requests for the available quota. The most common thing is for a request not to have a key parameter, which limits requests to 300 requests per 24 hour period (this quota of 300 also gets consumed when making requests with a key value, so they're gone very quickly). The URL in your report shows that a key value was provided, which provides a 10k request/24 hour period quota. FIRE doesn't make a lot of requests of the SE API. So, unless you have something else which is making a large number of requests, or are sharing your IP, then this is relatively unlikely.

2. Not obeying a backoff which was sent on another request. This is the most likely issue. It's unlikely FIRE can do much here to help. While FIRE is not currently all that great wrt. handling backoff, this is commonly a system level problem, not a userscript level problem. The throttling that SE does by providing a backoff is on a per IP/per endpoint basis. Once there are enough violations of the backoff throttling, SE imposes a complete throttle on the IP for a period of time. The problem is that to actually obey the backoff throttling correctly, you have to know about every single request made by the IP address. That's just not possible for a userscript, or even the browser itself. It is possible for a userscript to obey a backoff, particularly on a per-tab basis, and possible, but harder, on a per-userscript - per-browser profile basis. FIRE could do better in handling backoff, but for a userscript, it's not that bad. [I have written code to make it even better, and have ideas for even further improvement, but those are not yet implemented.]

What's probably happening is that you started Chrome after your upgrade. Chrome is then starting up a bunch of tabs to restore your session, or sessions, with each tab making independent requests of the SE API as part of starting up the tab's userscripts. So, even if the userscripts do obey a backoff that they receive, only the first one/couple/few will get an actual backoff response. SE will see the additional requests from tabs where the script doesn't know there's a backoff and think that the IP is just continuing to disobey the previous backoff response. For the next several requests, SE will provide just an error response. If the requests continue, then SE will impose an actual IP level block on the SE API, which will affect everything.

It's possible for the userscript to provide some cross-tab coordination for just that userscript, and, as I've said, I have ideas of how I might do that for FIRE and AIM (although mostly for the MS API, rather than the SE API). However, it's not possible for a userscript to coordinate this even with other arbitrary userscripts, let alone across every single use of the SE API from that IP (which could be across different systems all behind a router).