Rate limit issues

[seanwendt]

https://github.com/j0be/PowerDeleteSuite/blob/3feb575c0424394266fc0020681a69ae06f104c7/powerdeletesuite.js#L912

5 second timeout runs into issues, see an example header response on del call:

x-ratelimit-remaining: 1.0 // how many requests I can make before reset x-ratelimit-reset: 37 // how many seconds until reset x-ratelimit-used: 99 // how many requests I've made; another call will be made in 5 seconds, which will succeed, and another in 5 more which will fail.

100 requests per 10 minute period is the rate limit I am getting consistently, and with a deletion every 5 seconds (plus additional calls to next comment page) it's not enough time, eventually running into an error and getting the "Error deleting comment, would you like to retry?" popup, which stops all progress until 1) user clicks the button to retry && 2) rate limit has been reset

Recommend making this rate limit configurable for advanced users (lazy fix) or reading the response headers and acting accordingly. Happy to make a PR with your guidance if public contributions are welcomed.

[harrier1231]

Just found this tool and then saw your suggestion. I'm not able to use the tool but I believe in my non tech brain this might solve the issue.

[Lillecarl]

https://github.com/j0be/PowerDeleteSuite/commit/ac950305d9bdaa7f194bf343988bfbbae73f02b7

I'm not in a hurry, I just want my 2500+ comments gone eventually so i put a conservative 7 seconds instead of 5, keeps us well within their ratelimits.

@seanwendt Control flow seems weird so idk where i can read response headers, but that'd be the better solution indeed. Calculating requests left vs time to reset for interval making the flow smooth too.

[braboobssiere]

i'm testing this (i also change default timeout to 3 seconds, you can change it back to 5 if you encounter a problem or experiment with x-ratelimit-remaining number to account for searching api call). this is only for edit and del endpoints so it need to factored searching endpoints too (thus why it's 5)

delete: function (item) {
  if (pd.performActions) {
    $.ajax({
      url: "/api/del",
      method: "post",
      data: {
        id: item.data.name,
        executed: "deleted",
        uh: pd.config.uh,
        renderstyle: "html",
      },
      complete: function (xhr) {
        // Read headers
        var rateLimitRemaining = parseInt(xhr.getResponseHeader('x-ratelimit-remaining'), 10);
        var rateLimitReset = parseInt(xhr.getResponseHeader('x-ratelimit-reset'), 10);

        // Determine timeout
        var timeout = 3000; // default timeout
        if (rateLimitRemaining <= 5) {
          timeout = (rateLimitReset + 1) * 1000; 
        }

        // Set the next action with the calculated timeout
        setTimeout(() => {
          pd.task.items[0].pdDeleted = true;
          pd.actions.children.handleSingle();
        }, timeout);
      }
    }).fail(function () {
      pd.task.info.errors++;
      if (
        confirm(
          "Error deleting " +
            (item.kind == "t3" ? "post" : "comment") +
            ", would you like to retry?"
        )
      ) {
        pd.actions.children.handleSingle();
      } else {
        pd.actions.children.finishItem();
        pd.actions.children.handleGroup();
      }
    });
  } else {
    pd.task.items[0].pdDeleted = true;
    pd.task.after = pd.task.items[0].data.name;
    pd.actions.children.handleSingle();
  }
},

edit: function (item) {
  if (pd.performActions) {
    var editString = pd.task.config.editText ||
      pd.editStrings[Math.floor(Math.random() * pd.editStrings.length)];
    $.ajax({
      url: "/api/editusertext",
      method: "post",
      data: {
        thing_id: item.data.name,
        text: editString,
        id: "#form-" + item.data.name,
        r: item.data.subreddit,
        uh: pd.config.uh,
        renderstyle: "html",
      },
            complete: function (xhr) {
        // Read headers
        var rateLimitRemaining = parseInt(xhr.getResponseHeader('x-ratelimit-remaining'), 10);
        var rateLimitReset = parseInt(xhr.getResponseHeader('x-ratelimit-reset'), 10);

        // Determine timeout
        var timeout = 3000; // default timeout
        if (rateLimitRemaining <= 5) {
          timeout = (rateLimitReset + 1) * 1000; 
        }

        // Set the next action with the calculated timeout
        setTimeout(() => {
          pd.task.items[0].pdEdited = true;
          pd.actions.children.handleSingle();
        }, timeout);
      }
    }).fail(function () {
      pd.task.info.errors++;
      if (
        !confirm(
          "Error editing " +
            (item.kind == "t3" ? "post" : "comment") +
            ", would you like to retry?"
        )
      ) {
        item.pdEdited = true;
      }
      pd.actions.children.handleSingle();
    });
  } else {
    pd.task.items[0].pdEdited = true;
    pd.actions.children.handleSingle();
  }
},
  },
  ui: {

for ref here is response header:

accept-ranges: bytes
cache-control: private, s-maxage=0, max-age=0, must-revalidate, no-store
content-encoding: gzip
content-length: 2987
content-type: application/json; charset=UTF-8
date: Wed, 28 Aug 2024 07:02:32 GMT
expires: -1
nel: {"report_to": "w3-reporting-nel", "max_age": 14400, "include_subdomains": false, "success_fraction": 1.0, "failure_fraction": 1.0}
report-to: {"group": "w3-reporting-nel", "max_age": 14400, "include_subdomains": true,  "endpoints": [{ "url": "https://w3-reporting-nel.reddit.com/reports" }]}, {"group": "w3-reporting", "max_age": 14400, "include_subdomains": true, "endpoints": [{ "url": "https://w3-reporting.reddit.com/reports" }]}, {"group": "w3-reporting-csp", "max_age": 14400, "include_subdomains": true, "endpoints": [{ "url": "https://w3-reporting-csp.reddit.com/reports" }]}
server: snooserv
strict-transport-security: max-age=31536000; includeSubdomains
vary: accept-encoding
via: 1.1 varnish
x-content-type-options: nosniff
x-firefox-spdy: h2
x-frame-options: SAMEORIGIN
x-ratelimit-remaining: 87.0
x-ratelimit-reset: 447
x-ratelimit-used: 13
x-ua-compatible: IE=edge
x-xss-protection: 1; mode=block

[jfl88]

@braboobssiere I tried this myself and also tried your version and neither of them seemed to work, still hitting the Ratelimit-Remaining = 0 before it successfully throttles.

[braboobssiere]

@braboobssiere I tried this myself and also tried your version and neither of them seemed to work, still hitting the Ratelimit-Remaining = 0 before it successfully throttles.

edit2: now i add ratelimit to search endpoint too (on error of handle function) and it's works