同一模型多个APIKey随机轮询

[lopins]

同一模型多个APIKey随机轮询，每次随机一组

[fruitbars]

支持的哈，可以看下这个参数选项load_balancing

https://rsxlecz746.feishu.cn/wiki/MTjvwu1Ejio5ofkjnV0coviknad?fromScene=spaceOverview

[lopins]

支持的哈，可以看下这个参数选项load_balancing

https://rsxlecz746.feishu.cn/wiki/MTjvwu1Ejio5ofkjnV0coviknad?fromScene=spaceOverview

不是全局随机，而是单个节点下，我有两组不同的参数！ 如openai: sk-xxxxx,sk-yyyyy 简化配置,同格式同模型在同一父节点下

[fruitbars]

可以使用credential_list，参考《2.5 credentials和credential_list字段详解》 https://rsxlecz746.feishu.cn/wiki/D37WwNS4fiNnGLkmsu6cM76Nnkf?fromScene=spaceOverview

[lopins]

可以使用credential_list，参考《2.5 credentials和credential_list字段详解》 https://rsxlecz746.feishu.cn/wiki/D37WwNS4fiNnGLkmsu6cM76Nnkf?fromScene=spaceOverview

好的

{
  "server_port": ":9099",
  "log_level": "dev",
  "debug": false,
  "load_balancing": "random",
  "model_redirect": {
    "all": "4.0Ultra"
  },
  "services": {
    "xinghuo": [
      {
        "models": [
          "4.0Ultra",
          "spark-pro",
          "spark-max"
        ],
        "enabled": true,
        "credential_list": [
          {
            "appid": "xxx",
            "api_key": "xxx",
            "api_secret": "xxx"
          },
          {
            "appid": "xxx",
            "api_key": "xxx",
            "api_secret": "xxx"
          }
        ],
        "limit": {
          "qps": 2,
          "timeout": 10
        }
      }
    ]
  }
}