Use `claude-3-5-sonnet-20240620` from different providers/channels.

[hongyi-zhao]

If I want to use claude-3-5-sonnet-20240620 from different providers/channels with the following "model_map" method based config.json:

{
  "api_key": "123456",
  "debug": false,
  "load_balancing": "random",
  "proxy": {
    "type": "http",
    "http_proxy": "http://127.0.0.1:8080",
    "strategy": "default"
  },
  "services": {
    "claude.ai": [
      {
        "models": [
          "claude.ai/claude-3-5-sonnet-20240620"
        ],
        "enabled": true,
        "credentials": {
          "api_key": "sk-ant-xxx"
        },
        "model_map": {
          "claude.ai/claude-3-5-sonnet-20240620": "claude-3-5-sonnet-20240620"
        },        
        "server_url": "https://api.anthropic.com/v1/messages",
        "use_proxy": true
      }
    ],
    "wildcard.com.cn": [
      {
        "models": [
          "wildcard.com.cn/claude-3-5-sonnet-20240620"
        ],
        "enabled": true,
        "credentials": {
          "api_key": "sk-xxx"
        },
        "model_map": {
          "wildcard.com.cn/claude-3-5-sonnet-20240620": "claude-3-5-sonnet-20240620"
        },        
        "server_url": "https://api.gptsapi.net/v1/chat/completions"
      }
    ] 
  }
}

But the access to these models failed:

$ curl http://127.0.0.1:9090/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer 123456"   -d '{
  "model": "claude.ai/claude-3-5-sonnet-20240620",
  "messages": [{"role": "user", "content": "Hello, GPT-4!"}]
}'
{"error":"service handler not found"}

$ curl http://127.0.0.1:9090/v1/chat/completions   -H "Content-Type: application/json"   -H "Authorization: Bearer 123456"   -d '{
  "model": "wildcard.com.cn/claude-3-5-sonnet-20240620",
  "messages": [{"role": "user", "content": "Hello, GPT-4!"}]
}'
{"error":"service handler not found"}

On the other hand, how to do it with model_redirect?

[hongyi-zhao]

After trial and error, the following method work:

$ cat config_claude.json 
{
  "api_key": "123456",
  "debug": false,
  "load_balancing": "random",
  "proxy": {
    "type": "http",
    "http_proxy": "http://127.0.0.1:8080",
    "strategy": "default"
  },
  "services": {
    "claude": [
      {
        "models": [
          "claude/claude-3-5-sonnet-20240620"
        ],
        "enabled": true,
        "credentials": {
          "api_key": "sk-ant-xxx"
        },
        "model_redirect": {
          "claude.ai/claude-3-5-sonnet-20240620": "claude-3-5-sonnet-20240620"
        },        
        "server_url": "https://api.anthropic.com/v1/messages",
        "use_proxy": true
      }
    ],
    "openai": [
      {
        "models": [
          "my-claude-3-5-sonnet-20240620"
        ],
        "enabled": true,
        "credentials": {
          "api_key": "sk-xxx"
        },
        "model_redirect": {
          "my-claude-3-5-sonnet-20240620": "claude-3-5-sonnet-20240620"
        },        
        "server_url": "https://api.gptsapi.net/v1/chat/completions"
      }
    ] 
  }
}

In the above config file, model_redirect can also be replaced by model_map, and no other changes are needed.

The original failure was due to the following two reasons:

• The name of the services provider cannot be arbitrary. Each vendor has a corresponding name, aka, "claude" and "openai" should be used here instead of "claude.ai" and "wildcard.com.cn" used before.
• The syntax for "model_redirect" in the documentation example is incorrect. It should be written as shown above to work.