search

Aider-AI / aider

aider is AI pair programming in your terminal
https://aider.chat/
Apache License 2.0
21.17k stars 1.97k forks source link

Using Hyperbolic to as OPENAI_API_BASE - Getting Connection Error #1759

Closed akamalov closed 1 month ago

akamalov commented 1 month ago

Issue

Environment

Windows 11
conda 23.7.2
Python 3.11.9
aider 0.57.1

Export env.variables:

(aider) D:\SD> set OPENAI_API_KEY=<hyperbolic_api_key_here> (aider) D:\SD> set OPENAI_API_BASE=https://api.hypberbolic.xyz/v1/

Ran aider-chat as:

(aider) D:\SD\minesweeper>aider --model openai/Qwen/Qwen2-VL-7B-Instruct

Warning for openai/Qwen/Qwen2-VL-7B-Instruct: Unknown context window size and costs, using sane defaults.
For more info, see: https://aider.chat/docs/llms/warnings.html
You can skip this check with --no-show-model-warnings

Proceed anyway? (Y)es/(N)o [Yes]: Y
Aider v0.57.1
Model: openai/Qwen/Qwen2-VL-7B-Instruct with whole edit format
Git repo: ..\.git with 3 files
Repo-map: disabled
Use /help <question> for help, run "aider --help" to see cmd line args
Note: in-chat filenames are always relative to the git working dir, not the current working dir.
Cur working dir: D:\SD\minesweeper
Git working dir: D:\SD
>

Ran a prompt:

> Make a minesweeper game using html,css,js

litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 0.2 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 0.5 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 1.0 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 2.0 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 4.0 seconds...

I have no problem running it against OpenAI, it's with Hyperbolic it is causing a problem. I double-checked, API base for Hyperbolic is correct and aider should accept it, but no dice :(

Version and model info

No response

fry69 commented 1 month ago

Thank you for filing this issue.

Is openai/Qwen/Qwen2-VL-7B-Instruct the correct model name for Hyperbolic? If so, can you please start aider with the additional command line argument --verbose and try to decipher what is going wrong?

akamalov commented 1 month ago

Thank you for your response @fry69 . You're right, it looks like hyperbolic changed their model naming structure. However, I am getting the following:

(aider) D:\tmp>aider --model Qwen/Qwen2-VL-72B-Instruct --verbose
Config files search order, if no --config:
  - D:\tmp\.aider.conf.yml
  - C:\Users\akama\.aider.conf.yml
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Too soon to check version: 21.2 hours
Command Line Args:   --model Qwen/Qwen2-VL-72B-Instruct --verbose

Environment Variables:
  OPENAI_API_KEY:    ...lgAA
  OPENAI_API_BASE:   https://api.hypberbolic.xyz/v1/

Defaults:
  --model-settings-file:.aider.model.settings.yml
  --model-metadata-file:.aider.model.metadata.json
  --map-refresh:     auto
  --cache-keepalive-pings:0
  --map-multiplier-no-files:2
  --env-file:        D:\tmp\.env
  --input-history-file:D:\tmp\.aider.input.history
  --chat-history-file:D:\tmp\.aider.chat.history.md
  --user-input-color:#00cc00
  --tool-error-color:#FF2222
  --tool-warning-color:#FFA500
  --assistant-output-color:#0088ff
  --code-theme:      default
  --aiderignore:     D:\tmp\.aiderignore
  --lint-cmd:        []
  --test-cmd:        []
  --voice-language:  en
  --encoding:        utf-8

Option settings:
  - aiderignore: D:\tmp\.aiderignore
  - anthropic_api_key: None
  - apply: None
  - assistant_output_color: #0088ff
  - attribute_author: True
  - attribute_commit_message_author: False
  - attribute_commit_message_committer: False
  - attribute_committer: True
  - auto_commits: True
  - auto_lint: True
  - auto_test: False
  - cache_keepalive_pings: 0
  - cache_prompts: False
  - chat_history_file: D:\tmp\.aider.chat.history.md
  - chat_language: None
  - check_update: True
  - code_theme: default
  - commit: False
  - commit_prompt: None
  - config: None
  - dark_mode: False
  - dirty_commits: True
  - dry_run: False
  - edit_format: None
  - encoding: utf-8
  - env_file: D:\tmp\.env
  - exit: False
  - file: None
  - files: []
  - git: True
  - gitignore: True
  - gui: False
  - input_history_file: D:\tmp\.aider.input.history
  - install_main_branch: False
  - just_check_update: False
  - light_mode: False
  - lint: False
  - lint_cmd: []
  - list_models: None
  - llm_history_file: None
  - map_multiplier_no_files: 2
  - map_refresh: auto
  - map_tokens: None
  - max_chat_history_tokens: None
  - message: None
  - message_file: None
  - model: Qwen/Qwen2-VL-72B-Instruct
  - model_metadata_file: .aider.model.metadata.json
  - model_settings_file: .aider.model.settings.yml
  - openai_api_base: https://api.hypberbolic.xyz/v1/
  - openai_api_deployment_id: None
  - openai_api_key: ...lgAA
  - openai_api_type: None
  - openai_api_version: None
  - openai_organization_id: None
  - pretty: True
  - read: None
  - restore_chat_history: False
  - show_diffs: False
  - show_model_warnings: True
  - show_prompts: False
  - show_repo_map: False
  - stream: True
  - subtree_only: False
  - suggest_shell_commands: True
  - test: False
  - test_cmd: []
  - tool_error_color: #FF2222
  - tool_output_color: None
  - tool_warning_color: #FFA500
  - upgrade: False
  - user_input_color: #00cc00
  - verbose: True
  - verify_ssl: True
  - vim: False
  - voice_language: en
  - weak_model: None
  - yes: None

Checking imports for version 0.57.1 and executable D:\anaconda3\envs\aider\python.exe
Installs file: C:\Users\akama\.aider\installs.json
Installs file exists and loaded
Not first run, loading imports in background thread
No model settings files loaded
Searched for model settings files:
  - C:\Users\akama\.aider.model.settings.yml
  - D:\tmp\.aider.model.settings.yml
Model info:
{}
Warning for Qwen/Qwen2-VL-72B-Instruct: Unknown which environment variables are required.
Warning for Qwen/Qwen2-VL-72B-Instruct: Unknown context window size and costs, using sane defaults.
For more info, see: https://aider.chat/docs/llms/warnings.html
You can skip this check with --no-show-model-warnings

Proceed anyway? (Y)es/(N)o [Yes]: Yes
Aider v0.57.1
Model: Qwen/Qwen2-VL-72B-Instruct with whole edit format
Git repo: .git with 0 files
Repo-map: disabled
Use /help <question> for help, run "aider --help" to see cmd line args
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> Create sample python-based snake game

tokenizer.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 7.03M/7.03M [00:02<00:00, 3.44MB/s]

SYSTEM Act as an expert software developer.
SYSTEM Take requests for changes to the supplied code.
SYSTEM If the request is ambiguous, ask questions.
SYSTEM
SYSTEM Always reply to the user in the same language they are using.
SYSTEM
SYSTEM
SYSTEM Once you understand the request you MUST:
SYSTEM 1. Determine if any code changes are needed.
SYSTEM 2. Explain any needed changes.
SYSTEM 3. If changes are needed, output a copy of each file that needs changes.
SYSTEM
SYSTEM To suggest changes to a file you MUST return the entire content of the updated file.
SYSTEM You MUST use this *file listing* format:
SYSTEM
SYSTEM path/to/filename.js
SYSTEM ```
SYSTEM // entire file content ...
SYSTEM // ... goes in between
SYSTEM ```
SYSTEM
SYSTEM Every *file listing* MUST use this format:
SYSTEM - First line: the filename with any originally provided path
SYSTEM - Second line: opening ```
SYSTEM - ... entire content of the file ...
SYSTEM - Final line: closing ```
SYSTEM
SYSTEM To suggest changes to a file you MUST return a *file listing* that contains the entire content of the file.
SYSTEM *NEVER* skip, omit or elide content from a *file listing* using "..." or by adding comments like "... rest of code..."!
SYSTEM Create a new file you MUST return a *file listing* which includes an appropriate filename, including any appropriate path.
SYSTEM
SYSTEM

USER Change the greeting to be more casual

ASSISTANT Ok, I will:
ASSISTANT
ASSISTANT 1. Switch the greeting text from "Hello" to "Hey".
ASSISTANT
ASSISTANT show_greeting.py
ASSISTANT ```
ASSISTANT import sys
ASSISTANT
ASSISTANT def greeting(name):
ASSISTANT     print(f"Hey {name}")
ASSISTANT
ASSISTANT if __name__ == '__main__':
ASSISTANT     greeting(sys.argv[1])
ASSISTANT ```

USER I switched to a new code base. Please don't consider the above files or try to edit them any longer.

ASSISTANT Ok.

USER I am not sharing any files that you can edit yet.

ASSISTANT Ok.

USER Create sample python-based snake game

BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed
model=Qwen/Qwen2-VL-72B-Instruct
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more:
https://docs.litellm.ai/docs/providers
fry69 commented 1 month ago

Thank you for your reply.

Actually I was wrong, I forgot that you had to add openai/ to the model name for LiteLLM, see -> https://aider.chat/docs/llms/openai-compat.html

So the correct model name for aider should be ->

--model openai/Qwen/Qwen2-VL-72B-Instruct

Can you please re-run aider with this model name and --verbose?

akamalov commented 1 month ago 
(aider) D:\tmp>aider --model openai/Qwen/Qwen2-VL-72B-Instruct --verbose
Config files search order, if no --config:
  - D:\tmp\.aider.conf.yml
  - C:\Users\akama\.aider.conf.yml
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Too soon to check version: 21.4 hours
Command Line Args:   --model openai/Qwen/Qwen2-VL-72B-Instruct --verbose

Environment Variables:
  OPENAI_API_KEY:    ...lgAA
  OPENAI_API_BASE:   https://api.hypberbolic.xyz/v1/

Defaults:
  --model-settings-file:.aider.model.settings.yml
  --model-metadata-file:.aider.model.metadata.json
  --map-refresh:     auto
  --cache-keepalive-pings:0
  --map-multiplier-no-files:2
  --env-file:        D:\tmp\.env
  --input-history-file:D:\tmp\.aider.input.history
  --chat-history-file:D:\tmp\.aider.chat.history.md
  --user-input-color:#00cc00
  --tool-error-color:#FF2222
  --tool-warning-color:#FFA500
  --assistant-output-color:#0088ff
  --code-theme:      default
  --aiderignore:     D:\tmp\.aiderignore
  --lint-cmd:        []
  --test-cmd:        []
  --voice-language:  en
  --encoding:        utf-8

Option settings:
  - aiderignore: D:\tmp\.aiderignore
  - anthropic_api_key: None
  - apply: None
  - assistant_output_color: #0088ff
  - attribute_author: True
  - attribute_commit_message_author: False
  - attribute_commit_message_committer: False
  - attribute_committer: True
  - auto_commits: True
  - auto_lint: True
  - auto_test: False
  - cache_keepalive_pings: 0
  - cache_prompts: False
  - chat_history_file: D:\tmp\.aider.chat.history.md
  - chat_language: None
  - check_update: True
  - code_theme: default
  - commit: False
  - commit_prompt: None
  - config: None
  - dark_mode: False
  - dirty_commits: True
  - dry_run: False
  - edit_format: None
  - encoding: utf-8
  - env_file: D:\tmp\.env
  - exit: False
  - file: None
  - files: []
  - git: True
  - gitignore: True
  - gui: False
  - input_history_file: D:\tmp\.aider.input.history
  - install_main_branch: False
  - just_check_update: False
  - light_mode: False
  - lint: False
  - lint_cmd: []
  - list_models: None
  - llm_history_file: None
  - map_multiplier_no_files: 2
  - map_refresh: auto
  - map_tokens: None
  - max_chat_history_tokens: None
  - message: None
  - message_file: None
  - model: openai/Qwen/Qwen2-VL-72B-Instruct
  - model_metadata_file: .aider.model.metadata.json
  - model_settings_file: .aider.model.settings.yml
  - openai_api_base: https://api.hypberbolic.xyz/v1/
  - openai_api_deployment_id: None
  - openai_api_key: ...lgAA
  - openai_api_type: None
  - openai_api_version: None
  - openai_organization_id: None
  - pretty: True
  - read: None
  - restore_chat_history: False
  - show_diffs: False
  - show_model_warnings: True
  - show_prompts: False
  - show_repo_map: False
  - stream: True
  - subtree_only: False
  - suggest_shell_commands: True
  - test: False
  - test_cmd: []
  - tool_error_color: #FF2222
  - tool_output_color: None
  - tool_warning_color: #FFA500
  - upgrade: False
  - user_input_color: #00cc00
  - verbose: True
  - verify_ssl: True
  - vim: False
  - voice_language: en
  - weak_model: None
  - yes: None

Checking imports for version 0.57.1 and executable D:\anaconda3\envs\aider\python.exe
Installs file: C:\Users\akama\.aider\installs.json
Installs file exists and loaded
Not first run, loading imports in background thread
No model settings files loaded
Searched for model settings files:
  - C:\Users\akama\.aider.model.settings.yml
  - D:\tmp\.aider.model.settings.yml
Model info:
{}
Warning for openai/Qwen/Qwen2-VL-72B-Instruct: Unknown context window size and costs, using sane defaults.
For more info, see: https://aider.chat/docs/llms/warnings.html
You can skip this check with --no-show-model-warnings

Proceed anyway? (Y)es/(N)o [Yes]: Y
Aider v0.57.1
Model: openai/Qwen/Qwen2-VL-72B-Instruct with whole edit format
Git repo: .git with 0 files
Repo-map: disabled
Use /help <question> for help, run "aider --help" to see cmd line args
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> Create sample python-based snake game

SYSTEM Act as an expert software developer.
SYSTEM Take requests for changes to the supplied code.
SYSTEM If the request is ambiguous, ask questions.
SYSTEM
SYSTEM Always reply to the user in the same language they are using.
SYSTEM
SYSTEM
SYSTEM Once you understand the request you MUST:
SYSTEM 1. Determine if any code changes are needed.
SYSTEM 2. Explain any needed changes.
SYSTEM 3. If changes are needed, output a copy of each file that needs changes.
SYSTEM
SYSTEM To suggest changes to a file you MUST return the entire content of the updated file.
SYSTEM You MUST use this *file listing* format:
SYSTEM
SYSTEM path/to/filename.js
SYSTEM ```
SYSTEM // entire file content ...
SYSTEM // ... goes in between
SYSTEM ```
SYSTEM
SYSTEM Every *file listing* MUST use this format:
SYSTEM - First line: the filename with any originally provided path
SYSTEM - Second line: opening ```
SYSTEM - ... entire content of the file ...
SYSTEM - Final line: closing ```
SYSTEM
SYSTEM To suggest changes to a file you MUST return a *file listing* that contains the entire content of the file.
SYSTEM *NEVER* skip, omit or elide content from a *file listing* using "..." or by adding comments like "... rest of code..."!
SYSTEM Create a new file you MUST return a *file listing* which includes an appropriate filename, including any appropriate path.
SYSTEM
SYSTEM

USER Change the greeting to be more casual

ASSISTANT Ok, I will:
ASSISTANT
ASSISTANT 1. Switch the greeting text from "Hello" to "Hey".
ASSISTANT
ASSISTANT show_greeting.py
ASSISTANT ```
ASSISTANT import sys
ASSISTANT
ASSISTANT def greeting(name):
ASSISTANT     print(f"Hey {name}")
ASSISTANT
ASSISTANT if __name__ == '__main__':
ASSISTANT     greeting(sys.argv[1])
ASSISTANT ```

USER I switched to a new code base. Please don't consider the above files or try to edit them any longer.

ASSISTANT Ok.

USER I am not sharing any files that you can edit yet.

ASSISTANT Ok.

USER Create sample python-based snake game

litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 0.2 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 0.5 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 1.0 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 2.0 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 4.0 seconds...

^C again to exit
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
>
fry69 commented 1 month ago

This is strange, it should work, aider via LiteLLM does practically not much more than this example Python code -> https://docs.hyperbolic.xyz/docs/python-api

Can you please try to connect to the Hyperbolic API via curl to see the actual network errors? ->

curl -X POST "https://api.hyperbolic.xyz/v1/chat/completions" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $HYPERBOLIC_API_KEY" \
    -d '{
        "messages": [
            {
                "role": "system",
            "content": "You are a helpful and polite assistant."
            },
            {
                "role": "user",
                "content": "What is Chinese hotpot?"
            }
        ],
        "model": "Qwen/Qwen2-VL-72B-Instruct",
        "presence_penalty": 0,
        "temperature": 0.1,
        "top_p": 0.9,
        "stream": false
    }'

(directly taken form their website -> https://docs.hyperbolic.xyz/docs/rest-api only model name changed)

It works for me ->

{
  "id": "chat-5bf7967e7ae44bf293559d0c74f7b666",
  "object": "chat.completion",
  "created": 1727373969,
  "model": "Qwen/Qwen2-VL-72B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Chinese hotpot is a popular Chinese dish that involves cooking ingredients in a simmering pot of broth at the dining table. The broth is typically flavored with a variety of ingredients such as ginger, garlic, and chili peppers, and can be made with a variety of different types of broth, including chicken, beef, or vegetable broth.\nTo eat hotpot, diners typically sit around a table with a pot of broth in the center. They then add a variety of ingredients to the pot, such as meat, seafood, vegetables, and tofu, and cook them in the broth. The ingredients are typically served with a variety of dipping sauces, such as soy sauce, sesame oil, and chili oil.\nHotpot is a popular dish in China and is often enjoyed with friends and family as a social activity. It is also a popular dish in many other countries around the world, and there are many variations of hotpot that are enjoyed in different regions."
      },
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 27,
    "total_tokens": 218,
    "completion_tokens": 191
  }
}
akamalov commented 1 month ago

Yep. I do have the same output. May be it's because of Windows. I'll try to do it under Linux see if it works.

fry69 commented 1 month ago

Hmm. I can confirm this does not work for me either with aider, curl works of course as shown (macOS) -> 

$ aider --model openai/Qwen/Qwen2-VL-72B-Instruct

Aider v0.57.1
Model: openai/Qwen/Qwen2-VL-72B-Instruct with ask edit format
Git repo: .git with 6 files
Repo-map: disabled
Use /help <question> for help, run "aider --help" to see cmd line args

ask> Hello.                                                                                                                                                   

litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 0.2 seconds...
litellm.APIError: APIError: OpenAIException - Connection error.
Retrying in 0.5 seconds...
^C
akamalov commented 1 month ago

dang...

Nova-lotus commented 1 month ago

Sorry, I'm using hyperbolic, isn't the api base https://api.hyperbolic.xyz/v1/ not https://api.hypberbolic.xyz/v1/

I have an .env file with these values

AIDER_MODEL=openai/Qwen/Qwen2.5-72B-Instruct OPENAI_API_BASE=https://api.hyperbolic.xyz/v1/ OPENAI_API_KEY=

And I use qwen perfectly

fry69 commented 1 month ago

Thank you, I missed that typo and copy pasted the wrong API base URL.

fry69 commented 1 month ago

As this issue appears to be resolved, I'm closing it.

If any new related concerns arise, please feel free to comment, and I'll reopen the issue.

akamalov commented 1 month ago

Sorry, my bad!!!

On Sat, Sep 28, 2024, 11:37 AM fry69 @.***> wrote:

As this issue appears to be resolved, I'm closing it.

If any new related concerns arise, please feel free to comment, and I'll reopen the issue.

— Reply to this email directly, view it on GitHub https://github.com/paul-gauthier/aider/issues/1759#issuecomment-2380712140, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4FDSOBHFK2S2MTZB7TW2LZY3EMPAVCNFSM6AAAAABO5JPALOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBQG4YTEMJUGA . You are receiving this because you authored the thread.Message ID: @.***>