search

jacob-ai-bot / jacob

Just Another Coding Bot
https://jacb.ai
Apache License 2.0
131 stars 20 forks source link

JACoB PR for Issue Add Sonnet Haiku 3.5 model #251

Closed jacob-ai-bot[bot] closed 3 weeks ago

jacob-ai-bot[bot] commented 3 weeks ago

Summary:

Add the new claude-3-5-haiku-20241022 model. Pricing for Claude 3.5 Haiku starts at $1 per million input tokens and $5 per million output tokens. Just add it to the openAI and anthropic requests pages, not the front end chat model list. @jacob-ai-bot --skip-build

Plan:

Step 1: Edit /src/server/anthropic/request.ts

Task: Add 'claude-3-5-haiku-20241022' model to Anthropic request configurations

Instructions: In the file, add entries for the new 'claude-3-5-haiku-20241022' model in the constants 'CONTEXT_WINDOW', 'MAX_OUTPUT', 'INPUT_TOKEN_COSTS', and 'OUTPUT_TOKEN_COSTS'. Set the context window size to 200,000 tokens (matching other similar models). Set 'MAX_OUTPUT' to 8192 tokens. For 'INPUT_TOKEN_COSTS', set the cost to $1 per million tokens by assigning 1 divided by 1,000,000 (1 / 1,000,000). For 'OUTPUT_TOKEN_COSTS', set the cost to $5 per million tokens by assigning 5 divided by 1,000,000 (5 / 1,000,000). Ensure that all other logic in the file that depends on model names includes this new model where appropriate.

Exit Criteria: The 'claude-3-5-haiku-20241022' model is added to all relevant constants with correct values, the file compiles without errors, and the model can be used internally without issues.

Step 2: Edit /src/server/openai/request.ts

Task: Add 'claude-3-5-haiku-20241022' model to OpenAI request configurations

Instructions: In the file, add entries for the 'claude-3-5-haiku-20241022' model in the constants 'CONTEXT_WINDOW', 'MAX_OUTPUT', 'INPUT_TOKEN_COSTS', 'OUTPUT_TOKEN_COSTS', and 'PORTKEY_VIRTUAL_KEYS' if applicable. Assign the context window size as 200,000 tokens. Set 'MAX_OUTPUT' to 8192 tokens. For 'INPUT_TOKEN_COSTS', set the cost to $1 per million tokens (1 / 1,000,000). For 'OUTPUT_TOKEN_COSTS', set the cost to $5 per million tokens (5 / 1,000,000). If 'PORTKEY_VIRTUAL_KEYS' is used for model configuration, ensure the correct virtual key is assigned or set to an empty string if not applicable. Update any logic that selects or filters models to include this new model where necessary, without adding it to the front-end chat model list.

Exit Criteria: The 'claude-3-5-haiku-20241022' model is added to all relevant constants with correct values, the file compiles without errors, and internal processes that utilize this model function correctly.

jacob-ai-bot[bot] commented 3 weeks ago

Hello human! 👋

This PR was created by JACoB to address the issue Add Sonnet Haiku 3.5 model

Next Steps

  1. Please review the PR carefully. Auto-generated code can and will contain subtle bugs and mistakes.

  2. If you identify code that needs to be changed, please reject the PR with a specific reason. Be as detailed as possible in your comments. JACoB will take these comments, make changes to the code and push up changes. Please note that this process will take a few minutes.

  3. Once the code looks good, approve the PR and merge the code.