Sweep(slow): count tokens on the server side

Here's the PR! https://github.com/CNSeniorious000/free-chat/pull/20.

⚡ Sweep Basic Tier: I'm using GPT-4. You have 4 GPT-4 tickets left for the month and 3 for the day. (tracking ID: 1bfb7247a1)

For more GPT-4 tickets, visit our payment portal. For a one week free trial, try Sweep Pro (unlimited GPT-4 tickets).

Actions (click)

[ ] ↻ Restart Sweep

Sandbox Execution ✓

Here are the sandbox execution logs prior to making any changes:

Sandbox logs for 117c9ef

Checking src/pages/api/generate.ts for syntax errors... ✅ src/pages/api/generate.ts has no syntax errors! 1/1 ✓
Checking src/pages/api/generate.ts for syntax errors...
✅ src/pages/api/generate.ts has no syntax errors!

Sandbox passed on the latest endless, so sandbox checks will be enabled for this issue.

Step 1: 🔎 Searching

I found the following snippets in your repository. I will now analyze these snippets and come up with a plan.

Some code snippets I think are relevant in decreasing order of relevance (click to expand). If some file is missing from here, you can mention the path in the ticket description.

https://github.com/CNSeniorious000/free-chat/blob/117c9ef603e6b267a13c3b7c633fcb86624dbef8/src/components/Generator.tsx#L1-L19 https://github.com/CNSeniorious000/free-chat/blob/117c9ef603e6b267a13c3b7c633fcb86624dbef8/src/pages/api/generate.ts#L1-L15 https://github.com/CNSeniorious000/free-chat/blob/117c9ef603e6b267a13c3b7c633fcb86624dbef8/src/components/Generator.tsx#L217-L260 https://github.com/CNSeniorious000/free-chat/blob/117c9ef603e6b267a13c3b7c633fcb86624dbef8/src/utils/tiktoken.ts#L1-L38

Step 2: ⌨️ Coding

[X] Create src/utils/tiktoken-server.ts ✓ https://github.com/CNSeniorious000/free-chat/commit/09d72442ebe25ea72693afd406fe601d703d1b27

Create src/utils/tiktoken-server.ts with contents:
• Create a new utility file named `tiktoken-server.ts` in the `src/utils` directory for the server-side token counting logic.
• Use `tiktoken-js` instead of `tiktoken` as the server-side equivalent library.
• Define and export a function `countTokensServer` that implements the same logic as `countTokens` from `src/utils/tiktoken.ts`.
• Ensure the function interface matches that of the `countTokens` presently on the client side, taking an encoder and a list of messages as arguments and returning an object with the total token count.
• Make sure to wrap any initializations that are not available on the server, such as fetching base configurations or initializing WebAssembly modules, in a server-compatible manner.
[X] Running GitHub Actions for src/utils/tiktoken-server.ts ✓

Check src/utils/tiktoken-server.ts with contents:

Ran GitHub Actions for 09d72442ebe25ea72693afd406fe601d703d1b27:
• Vercel Preview Comments: ✓

[X] Modify src/pages/api/generate.ts ✓ https://github.com/CNSeniorious000/free-chat/commit/30a5ea4d0bdc06e092563c96327c3e11eeb3cff2

Modify src/pages/api/generate.ts with contents:
• In the `post` method of the API route, import the `countTokensServer` function from `src/utils/tiktoken-server.ts`.
• After retrieving the request body, apply the token counting logic to trim the `messages` array, ensuring it remains under a defined token limit.
• Use the constants defined in `src/components/Generator.tsx` like `minMessages` and `maxTokens` to set the lower message limit and token count limit. These may need to be moved to a shared constants file if they are not already.
• Ensure that after implementing the logic, the trimmed `messages` are then passed on for the rest of the processing where the generation payload is created.
[X] Running GitHub Actions for src/pages/api/generate.ts ✓

Check src/pages/api/generate.ts with contents:

Ran GitHub Actions for 30a5ea4d0bdc06e092563c96327c3e11eeb3cff2:
• Vercel Preview Comments: ✓

Step 3: 🔁 Code Review

I have finished reviewing the code for completeness. I did not find errors for sweep/server-side-token-counting_1.

🎉 Latest improvements to Sweep:

Sweep uses OpenAI's latest Assistant API to plan code changes and modify code! This is 3x faster and significantly more reliable as it allows Sweep to edit code and validate the changes in tight iterations, the same way as a human would.
Sweep now uses the rope library to refactor Python! Check out Large Language Models are Bad at Refactoring Code. To have Sweep refactor your code, try sweep: Refactor <your_file>.py!

💡 To recreate the pull request edit the issue title or description. To tweak the pull request, leave a comment on the pull request. ^{Join Our Discord}

CNSeniorious000 / free-chat