SyntaxError: Unexpected token

galpt commented 1 month ago

Got this error today. Everything worked fine several days ago.

Could not create "CGPS List - Chunk 1" - SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON
undefined:1
<!DOCTYPE html>
^

SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON
    at JSON.parse (<anonymous>)
    at parseJSONFromBytes (node:internal/deps/undici/undici:4306:19)
    at successSteps (node:internal/deps/undici/undici:4288:27)
    at fullyReadBody (node:internal/deps/undici/undici:2724:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async consumeBody (node:internal/deps/undici/undici:4297:7)
    at async request (file:///D:/katheryne/cf/cloudflare-gateway-pihole-scripts-main/lib/helpers.js:107:16)
    at async createZeroTrustListsOneByOne (file:///D:/katheryne/cf/cloudflare-gateway-pihole-scripts-main/lib/api.js:48:7)
    at async file:///D:/katheryne/cf/cloudflare-gateway-pihole-scripts-main/cf_list_create.js:141:5

mrrfv commented 1 month ago

I can reproduce this:

Text response: <!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Access denied | api.cloudflare.com used Cloudflare to restrict access</title>
...
        <h2 class="text-gray-600 leading-1.3 text-3xl lg:text-2xl font-light">You are being rate limited</h2>
      </header>

      <section class="w-240 lg:w-full mx-auto mb-8 lg:px-8">
          <div id="what-happened-section" class="w-1/2 md:w-full">
            <h2 class="text-3xl leading-tight font-normal mb-4 text-black-dark antialiased" data-translate="what_happened">What happened?</h2>
            <p>The owner of this website (api.cloudflare.com) has banned you temporarily from accessing this website.</p>

This is a rate limit from Cloudflare, which shouldn't happen unless fast mode is used:

The global rate limit for the Cloudflare API is 1200 requests per five minutes per user, and applies cumulatively regardless of whether the request is made via the dashboard, API key, or API token.

Very unusual. I guess this could be fixed by aggressively sleeping between requests, but that is a stopgap solution since the real problem lies within a misconfiguration on Cloudflare's side. Let's give them some time to fix this problem.

galpt commented 4 weeks ago

Yeah it's better if Cloudflare's aware of this problem.

While we're waiting for a stable fix from their side, for anyone who's looking for a temporary fix, I've tested updating filters from OISD Big to AdGuard DNS filter and didn't trigger the rate limit by modifying the /lib/api.js file, adding a 3-second sleep.

/**
 * Creates Zero Trust lists sequentially.
 * @param {string[]} items The domains.
 */
export const createZeroTrustListsOneByOne = async (items) => {
  let totalListNumber = Math.ceil(items.length / LIST_ITEM_SIZE);

  for (let i = 0, listNumber = 1; i < items.length; i += LIST_ITEM_SIZE) {
    const chunk = items
      .slice(i, i + LIST_ITEM_SIZE)
      .map((item) => ({ value: item }));
    const listName = `CGPS List - Chunk ${listNumber}`;

    try {
      await createZeroTrustList(listName, chunk);
      totalListNumber--;
      listNumber++;
      console.log(`Created "${listName}" list - ${totalListNumber} left`);
      await new Promise((r) => setTimeout(r, 3000)); // sleep for 3 secs
    } catch (err) {
      console.error(`Could not create "${listName}" - ${err.toString()}`);
      throw err;
    }
  }
};

/**
 * Deletes Zero Trust lists sequentially.
 * @param {Object[]} lists The lists to be deleted.
 * @param {number} lists[].id The ID of a list.
 * @param {string} lists[].name The name of a list.
 */
export const deleteZeroTrustListsOneByOne = async (lists) => {
  let totalListNumber = lists.length;

  for (const { id, name } of lists) {
    try {
      await deleteZeroTrustList(id);
      totalListNumber--;
      console.log(`Deleted ${name} list - ${totalListNumber} left`);
      await new Promise((r) => setTimeout(r, 3000)); // sleep for 3 secs
    } catch (err) {
      console.error(`Could not delete ${name} - ${err.toString()}`);
      throw err;
    }
  }
};

I guess it's good to leave this issue open for a while at least until Cloudflare's implemented a fix from their side.

mrrfv / cloudflare-gateway-pihole-scripts

SyntaxError: Unexpected token #111