search

jorenn92 / Maintainerr

Looks and smells like Overseerr, does the opposite. Maintenance tool for the Plex ecosystem
https://maintainerr.info
MIT License
751 stars 13 forks source link

Unable to add rule due to rate limiting #1211

Open sameleff opened 1 month ago

sameleff commented 1 month ago

Describe the bug Rule execution fails and the container automatically re-starts without adding any media items to the "Leaving Soon" collection. The same rule-sets work with the TV Show library (<100 shows), but do not work on the Movie library (~ 1,000 movies).

To Reproduce Steps to reproduce the behavior:

  1. Create a new rule
  2. Click Run Rules in the GUI
  3. Watch logs for data: 'Rate limit exceeded!'

Expected behavior Media items are added to collection even for large(r) libraries.

Screenshots image image

Device (please complete the following information):

Additional context Prior to clicking Run Rules (and after, for that matter), I confirmed that my Plex connection was working correctly by clicking "Test Saved" in the GUI. I am connecting locally (via internal IP). Same error regardless of whether I use SSL.

Is this expected behavior? I know my library isn't small, but I also didn't expect to experience rate-limit issues so soon. Is the rate-limit determined by the library size, or just the number of items added to a collection during a particular execution of the code? If the former, are there any workarounds? If library size is the driver, Maintainerr's use case will suffer since it will primarily be used by folks with large(r) libraries that don't want to maintain them manually. If the latter, what is the maximum number of items Maintainerr can add to a collection at one time?

jorenn92 commented 1 month ago

Hey, thanks for your report! There are already mitigations in place to avoid rate limit issues, but it's possible some limits changed and/or new once are implemented on Plex's side. Do you happen to still have the full log? I'd like to know what function is throwing the error.

It's odd that the server crashes after receiving this error, this makes me suspect it's a NextJS error instead. But i can't know for sure without the full log.

sameleff commented 3 weeks ago

Here are the logs.

As a note: even though I set DEBUG=true, the log file from maintainerr does not include these errors (I re-created them with the debug variable set to both false and true and the logs behave the same). I can, however, watch the logs in real-time via the logs command from the unraid webgui, and I was able to find these buried in the unraid OS files.

maintainerr_logs_08192024.log

P.S. I redacted my Plex token, but I'm not sure if there is anything else I should remove from this file.

jorenn92 commented 2 weeks ago

The logs indicate that the rate limits for the 'GetWatchlistHub' endpoint have been exceeded. Are you using the 'watchlisted by' rule? This feature is still considered 'experimental' (and unreleased) to help identify and resolve issues like this.

sameleff commented 2 weeks ago

Yes, I am using the 'watchlisted by' rule to track watchlists of 3 users. I use the same rule set (e.g. 3x 'watchlisted by' rules for each of the 3 users) to create a "Leaving Soon" collection from my TV Show library and the rule processes w/out errors. My movie library is definitely larger, but I still wouldn't have expected to encounter a rate limit issue. Do you need any other information to help troubleshoot?

jorenn92 commented 2 weeks ago

Requests to plex.tv are cached on Maintainer's side, but this cache is only held for 5 minutes. I’ve increased the caching duration to 10 minutes in hopes of resolving your issue. While I'm not sure about the exact rate limits for plex.tv, I assume they're quite high since the error only occurred 4 hours into the run. It's possible that we eventually exceeded the maximum number of allowed requests per hour.

You can test this with the main/develop docker image in about 30 minutes, once the images have been build.

sameleff commented 2 weeks ago

I updated to the latest develop branch (51786c9), but I'm still receiving the same error. If it helps, the error occurs within 30-60 seconds of clicking Run Rules from the webgui.

Adrian-Rasmussen commented 1 week ago

I am also facing the same issue, using the 2.1.0 release with the 'watchlisted by' rule

jorenn92 commented 1 week ago

A potential workaround was pushed for issue https://github.com/jorenn92/Maintainerr/issues/1250.