Implement token counting on server side (✓ Sandbox Passed)

[sweep-ai[bot]]

PR Feedback: 👎

Description

This pull request implements token counting on the server side.

Summary

• Updated src/pages/api/generate.ts to include token counting logic.
• Added src/utils/tiktoken-server.ts to handle token counting and initialization of Tiktoken.

Fixes #17.

---

🎉 Latest improvements to Sweep:

• Sweep uses OpenAI's latest Assistant API to plan code changes and modify code! This is 3x faster and significantly more reliable as it allows Sweep to edit code and validate the changes in tight iterations, the same way as a human would.
• Sweep now uses the rope library to refactor Python! Check out Large Language Models are Bad at Refactoring Code. To have Sweep refactor your code, try sweep: Refactor <your_file>.py!

---

💡 To get Sweep to edit this pull request, you can:

• Comment below, and Sweep can edit the entire PR
• Comment on a file, Sweep will only modify the commented file
• Edit the original issue to get Sweep to recreate the PR from scratch

[sweep-ai[bot]]

Sandbox Executions

• [X] Running GitHub Actions for src/utils/tiktoken-server.ts ✓

  Check src/utils/tiktoken-server.ts with contents:

Ran GitHub Actions for 09d72442ebe25ea72693afd406fe601d703d1b27:
• Vercel Preview Comments: ✓

• [X] Running GitHub Actions for src/pages/api/generate.ts ✓

  Check src/pages/api/generate.ts with contents:

Ran GitHub Actions for 30a5ea4d0bdc06e092563c96327c3e11eeb3cff2:
• Vercel Preview Comments: ✓

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
44444444444	❌ Failed (Inspect)			Dec 23, 2023 2:11pm
emphasize	❌ Failed (Inspect)			Dec 23, 2023 2:11pm
endless-chat	❌ Failed (Inspect)			Dec 23, 2023 2:11pm
free-chat	❌ Failed (Inspect)			Dec 23, 2023 2:11pm
free-chat-personal	❌ Failed (Inspect)			Dec 23, 2023 2:11pm

[sweep-ai[bot]]

Apply Sweep Rules to your PR?

• [ ] Apply: All new business logic should have corresponding unit tests.
• [ ] Apply: Refactor large functions to be more modular.
• [ ] Apply: Add docstrings to all functions and file headers.

[senior-dev-bot[bot]]

Hi there! :wave: Thanks for opening a PR. :tada: To get the most out of Senior Dev, please sign up in our Web App, connect your GitHub account, and add/join your organization CNSeniorious000. After that, you will receive code reviews beginning on your next opened PR. :rocket:

[netlify[bot]]

❌ Deploy Preview for endless-chat failed.

Name	Link
🔨 Latest commit	44b8b8a284c1db023694b31530e985272d082859
🔍 Latest deploy log	https://app.netlify.com/sites/endless-chat/deploys/6580a67726c50f00089cd7fb

[pr-explainer-bot[bot]]

Pull Request Report

Hey there! I've created a report for the pull request based on the commit history. Let's dive in!

Changes

1. Updated src/pages/api/generate.ts:
   • Added import for countTokensServer from @/utils/tiktoken-server.
   • Added minMessages and maxTokens constants.
   • Implemented token counting and trimming logic for messages.
   • Updated initOptions with trimmed messages.
   • Added error handling for token counting and trimming logic.

Suggestions

• In src/pages/api/generate.ts, consider extracting the token counting and trimming logic into a separate function for better readability and maintainability. Here's a suggested refactoring:

// Add this function at the top of the file
const trimMessages = (messages: ChatMessage[], maxTokens: number, minMessages: number) => {
  let trimmedMessages = [...messages];
  let tokenCount = countTokensServer(null, trimmedMessages).total;
  while (tokenCount > maxTokens && trimmedMessages.length > minMessages) {
    trimmedMessages.shift();
    tokenCount = countTokensServer(null, trimmedMessages).total;
  }
  return trimmedMessages;
};

// Update the post function
export const post: APIRoute = async ({ request }) => {
  // ...
  const trimmedMessages = trimMessages(messages, maxTokens, minMessages);
  // ...
};

Bugs

• No bugs found.

Improvements

• No improvements found.

Rating

I would rate the code a 7 out of 10. The code is generally readable and performs well. However, there are a few areas that could be refactored for better readability, such as extracting the token counting and trimming logic into a separate function. Overall, the code seems to be secure and error handling is in place.

That's it for the pull request report! Let me know if you need any further assistance. Have a great day! 😄

[netlify[bot]]

❌ Deploy Preview for chat-for-free failed.

Name	Link
🔨 Latest commit	44b8b8a284c1db023694b31530e985272d082859
🔍 Latest deploy log	https://app.netlify.com/sites/chat-for-free/deploys/6580a677ca656500085be314