search

jacob-ai-bot / jacob

Just Another Coding Bot
https://jacb.ai
Apache License 2.0
131 stars 20 forks source link

JACoB PR for Issue Add Sonnet Haiku 3.5 model #253

Closed jacob-local-kevin[bot] closed 3 weeks ago

jacob-local-kevin[bot] commented 3 weeks ago

Summary:

Add the new claude-3-5-haiku-20241022 model. Pricing for Claude 3.5 Haiku starts at $1 per million input tokens and $5 per million output tokens. Just add it to the openAI and anthropic requests pages, not the front end chat model list. @jacob-ai-bot --skip-build

Plan:

Step 1: Edit /src/server/anthropic/request.ts

Task: Add 'claude-3-5-haiku-20241022' model to the Anthropic request configurations

Instructions: In the file '/src/server/anthropic/request.ts', add the new model 'claude-3-5-haiku-20241022' to the following constants:

  1. In 'CONTEXT_WINDOW', add: 'claude-3-5-haiku-20241022': 200000,

  2. In 'MAX_OUTPUT', add: 'claude-3-5-haiku-20241022': 4096,

  3. In 'INPUT_TOKEN_COSTS', add: 'claude-3-5-haiku-20241022': 1 / ONE_MILLION,

  4. In 'OUTPUT_TOKEN_COSTS', add: 'claude-3-5-haiku-20241022': 5 / ONE_MILLION,

Ensure that the 'Model' type automatically includes the new model via keyof typeof CONTEXT_WINDOW.

Exit Criteria: The 'claude-3-5-haiku-20241022' model is added to all relevant constants in '/src/server/anthropic/request.ts', and the code compiles without errors.

Step 2: Edit /src/server/openai/request.ts

Task: Add 'claude-3-5-haiku-20241022' model configurations to OpenAI request configurations

Instructions: In the file '/src/server/openai/request.ts', add the new model 'claude-3-5-haiku-20241022' to the following constants:

  1. In 'CONTEXT_WINDOW', add: 'claude-3-5-haiku-20241022': 200000,

  2. In 'MAX_OUTPUT', add: 'claude-3-5-haiku-20241022': 4096,

  3. In 'INPUT_TOKEN_COSTS', add: 'claude-3-5-haiku-20241022': 1 / ONE_MILLION,

  4. In 'OUTPUT_TOKEN_COSTS', add: 'claude-3-5-haiku-20241022': 5 / ONE_MILLION,

  5. In 'PORTKEY_VIRTUAL_KEYS', add: 'claude-3-5-haiku-20241022': process.env.PORTKEY_VIRTUAL_KEY_ANTHROPIC,

In the sendGptRequest function, check if any special handling is required for 'claude-3-5-haiku-20241022'. If necessary, update the function to handle this model similarly to 'claude-3-5-sonnet-20241022'.

Exit Criteria: The 'claude-3-5-haiku-20241022' model is added to all relevant constants and mappings in '/src/server/openai/request.ts', any necessary handling in functions is updated, and the code compiles without errors.

jacob-local-kevin[bot] commented 3 weeks ago

Hello human! 👋

This PR was created by JACoB to address the issue Add Sonnet Haiku 3.5 model

Next Steps

  1. Please review the PR carefully. Auto-generated code can and will contain subtle bugs and mistakes.

  2. If you identify code that needs to be changed, please reject the PR with a specific reason. Be as detailed as possible in your comments. JACoB will take these comments, make changes to the code and push up changes. Please note that this process will take a few minutes.

  3. Once the code looks good, approve the PR and merge the code.