Error working with large files - Gemini

[Bruno-Alumn]

So I'm trying to de-obfuscate a file that has 3781204 characters. I know its quite a bit... I decided to go with Gemini due to the disclaimer of speed with OpenAI. Firstly the progress is incredibly slow (probably due to the file size). Anyways, after running this command and two hours (getting to 3% completion)

humanify gemini --apiKey="someKey" original.js

I get this error:

Processing file 1/1
Processing: 2%file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/@google/generative-ai/dist/index.mjs:401
    throw new GoogleGenerativeAIFetchError(⁠ Error fetching from ${url.toString()}: [${response.status} ${response.statusText}] ${message} ⁠, response.status, response.statusText, errorDetails);
          ^

GoogleGenerativeAIFetchError: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent: [503 Service Unavailable] The model is overloaded. Please try again later.
    at handleResponseNotOk (file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/@google/generative-ai/dist/index.mjs:401:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async makeRequest (file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/@google/generative-ai/dist/index.mjs:374:9)
    at async generateContent (file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/node_modules/@google/generative-ai/dist/index.mjs:817:22)
    at async file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/dist/index.mjs:56818:24
    at async visitAllIdentifiers (file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/dist/index.mjs:56604:21)
    at async file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/dist/index.mjs:56810:12
    at async unminify (file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/dist/index.mjs:202:27)
    at async Command.<anonymous> (file:///C:/Users/user/AppData/Roaming/npm/node_modules/humanifyjs/dist/index.mjs:56859:3) {
  status: 503,
  statusText: 'Service Unavailable',
  errorDetails: undefined
}

Node.js v18.20.4

Now from what I understand this is completely google's issue as stated in this thread. The top answer does suggest implementing a "back-off" feature, similar to rate-limit prevention techniques. Is it possible to do this? or is there another cause to the issue to do with my machine? tysm

[0xdevalias]

Related issues:

• https://github.com/jehna/humanify/issues/2

---

The top answer does suggest implementing a "back-off" feature, similar to rate-limit prevention techniques. Is it possible to do this? or is there another cause to the issue to do with my machine?

@Bruno-Alumn Based on a quick skim of the error + StackOverflow page, I think you're right in thinking this is more of a problem with Google + humanify not having a good 'error recovery mechanism' rather than anything specific with your machine.

[Bruno-Alumn]

Thank you very much for the reply @0xdevalias! Ill look into contributing to the project to solve the issue though it seems to have already been mentioned in the thread you linked. Either way, for my current situation, I decided to use OpenAI. It is true, the speed it noticeably slower, though i still had one question: Does the following seem like an accurate ratio between input and output tokens?

[0xdevalias]

Does the following seem like an accurate ratio between input and output tokens?

@Bruno-Alumn I don't really know what the ratios should look like specifically.. but I would definitely expect input to be MUCH higher, as that's going to be providing all of the code being referenced; whereas output should just be essentially giving the variable renames back; so that doesn't look overly surprising to me.

[0xdevalias]

I'm trying to process a pretty huge file and just ran into this:

RateLimitError: 429 Rate limit reached for gpt-4o-mini in organization org-abcdefghijklmnopqrstuvwx on requests per day (RPD): Limit 10000, Used 10000

I'm going to see about improving the rate limiting here:

// /src/plugins/openai/openai-rename.ts
+import Bottleneck from "bottleneck/light";

+// Math.floor(10_000 / 24) requests/hour
+const limiter = new Bottleneck({
+ "reservoir": Math.floor(10_000 / 24),
+ "reservoirRefreshAmount": Math.floor(10_000 / 24),
+ "reservoirRefreshInterval": 3_600_000
+});

export function openaiRename({
  apiKey,
  baseURL,
  model,
  contextWindowSize
}: {
  apiKey: string;
  baseURL: string;
  model: string;
  contextWindowSize: number;
}) {
  const client = new OpenAI({ apiKey, baseURL });

+  const wrapped = limiter.wrap(async (code: string): Promise<string> => {
    return await visitAllIdentifiers(
      code,
      async (name, surroundingCode) => {
        verbose.log(`Renaming ${name}`);
        verbose.log("Context: ", surroundingCode);

        const response = await client.chat.completions.create(
          toRenamePrompt(name, surroundingCode, model)
        );
        const result = response.choices[0].message?.content;
        if (!result) {
          throw new Error("Failed to rename", { cause: response });
        }
        const renamed = JSON.parse(result).newName;

        verbose.log(`Renamed to ${renamed}`);

        return renamed;
      },
      contextWindowSize,
      showPercentage
    );
+  });

+  return wrapped();
}

Originally posted by @brianjenkins94 in https://github.com/jehna/humanify/issues/167#issuecomment-2430584145