search

alibaba / higress

🤖 AI Gateway | AI Native API Gateway
https://higress.io
Apache License 2.0
2.9k stars 478 forks source link

使用 Higress + NextChat 快速搭建私人GPT助理 #938

Open johnlanni opened 4 months ago

johnlanni commented 4 months ago

第一步：创建一个名为 docker-compose.yml 的文件，并填入以下内容：

注意：

  1. YOUR_DASHSCOPE_API_KEY 需要替换为你自己的通义千问的 API Key；
  2. /path-to-local-config-folder 需要替换为一个本地可以用来保存配置文件的目录路径。
  3. CODE=访问密码 这里换成你自己的密码
version: '3.9'

networks:
  higress-net:
    external: false

services:
  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0
    environment:
      - CONFIG_TEMPLATE=ai-proxy
      - DEFAULT_AI_SERVICE=qwen
      - DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEY
    networks:
      - higress-net
    ports:
      - "8080:8080/tcp" # 如果本地 8080 端口被占用，那么可以修改冒号前面的端口号进行规避。冒号后的 8080 请勿修改
      - "8001:8001/tcp"
    volumes:
      - /path-to-local-config-folder:/data
    restart: always
  nextchat:
    image: yidadaa/chatgpt-next-web
    environment:
      - CODE=访问密码
      - OPENAI_API_KEY=unused
      - CUSTOM_MODELS=-all,+gpt-4o
      - BASE_URL=http://higress:8080 # 这一行请勿修改
    networks:
      - higress-net
    ports:
      - "3000:3000/tcp"
    restart: always

第二步：在命令行中运行以下命令，启动 docker compose 项目：

docker compose -p higress-ai up -d

第三步：在浏览器里访问 http://localhost:3000/，打开 NextChat 页面；

image

第四步：点击对话输入框工具栏最右侧的模型设置按钮，将模型切换为 gpt-4-turbo 并关闭窗口。

因为 Higress 的 AI Proxy 插件（可以访问 http://localhost:8001 查看插件配置）配置了 gpt-4-turbo 到 qwen-max 的模型映射，所以实际上这里提供的模型服务是 qwen-max

image

完成！现在你就可以与 AI 进行对话了。

按照上面文件进行配置，默认和你对话的将是通义千问。如果想切换到OpenAI ChatGPT，只需要将文件中的 DEFAULT_AI_SERVICE=qwen 修改为 DEFAULT_AI_SERVICE=openai，并添加 OpenAI 的 API key 配置，然后重新启动 docker compose 项目即可。

# 以上配置省略
  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.3.6
    environment:
      - CONFIG_TEMPLATE=ai-proxy
      - DEFAULT_AI_SERVICE=openai
      - OPENAI_API_KEY=YOUR_OPENAI_API_KEY
# 以下配置省略
docker compose -p higress-ai down
docker compose -p higress-ai up -d
Dwayneten commented 4 months ago

一直返回

{
  "error": true,
  "message": "empty response from server"
}

因为 Higress 的 AI Proxy 插件（可以访问 http://localhost:8001/ 查看插件配置）配置了 gpt-4-turbo 到 qwen-max 的模型映射，所以实际上这里提供的模型服务是 qwen-max

这个 AI Proxy 插件需要自己配置吗

CH3CHO commented 4 months ago

token 配的对吗？

Kent Dong

@. | ---- 回复的原邮件 ---- | 发件人 | @.> | | 发送日期 | 2024年05月17日 21:45 | | 收件人 | @.> | | 抄送人 | Subscribed @.> | | 主题 | Re: [alibaba/higress] 使用 Higress + NextChat 快速搭建私人GPT助理 (Issue #938) |

一直返回

{ "error": true, "message": "empty response from server" }

因为 Higress 的 AI Proxy 插件（可以访问 http://localhost:8001/ 查看插件配置）配置了 gpt-4-turbo 到 qwen-max 的模型映射，所以实际上这里提供的模型服务是 qwen-max

这个 AI Proxy 插件需要自己配置吗

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

Dwayneten commented 4 months ago

token 配的对吗？

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

CH3CHO commented 4 months ago

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

你可以在 docker-compose.yml 里给 higress 加一个 GATEWAY_COMPONENT_LOG_LEVEL=misc:error,wasm:debug 的环境变量，然后 exec 到 higress 容器里面，查看 /var/log/higress/gateway.log 文件，看看 DashScope 实际返回了什么。

Dwayneten commented 4 months ago

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

你可以在 docker-compose.yml 里给 higress 加一个 GATEWAY_COMPONENT_LOG_LEVEL=misc:error,wasm:debug 的环境变量，然后 exec 到 higress 容器里面，查看 /var/log/higress/gateway.log 文件，看看 DashScope 实际返回了什么。

有个这个错误

2024-05-18T01:57:11.528002Z     error   wasm    cannot fetch Wasm module oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:1.0.0: could not fetch Wasm OCI image: could not fetch image: Get "https://dockerauth.cn-hangzhou.aliyuncs.com/auth?scope=repository%3Aplugins%2Fai-proxy%3Apull&service=registry.aliyuncs.com%3Acn-hangzhou%3Achina%3Acri-r0xfyoxudmtseqq7": dial tcp [2408:4005:1000:10::2]:443: connect: network is unreachable
CH3CHO commented 4 months ago

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

你可以在 docker-compose.yml 里给 higress 加一个 GATEWAY_COMPONENT_LOG_LEVEL=misc:error,wasm:debug 的环境变量，然后 exec 到 higress 容器里面，查看 /var/log/higress/gateway.log 文件，看看 DashScope 实际返回了什么。

有个这个错误

2024-05-18T01:57:11.528002Z     error   wasm    cannot fetch Wasm module oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:1.0.0: could not fetch Wasm OCI image: could not fetch image: Get "https://dockerauth.cn-hangzhou.aliyuncs.com/auth?scope=repository%3Aplugins%2Fai-proxy%3Apull&service=registry.aliyuncs.com%3Acn-hangzhou%3Achina%3Acri-r0xfyoxudmtseqq7": dial tcp [2408:4005:1000:10::2]:443: connect: network is unreachable

拉不到插件镜像。你的机器能访问外网吗？

johnlanni commented 4 months ago

貌似是访问阿里云acr镜像走了ipv6导致的

image

看了下这个域名 higress-registry.cn-hangzhou.cr.aliyuncs.com 并没有 ipv6 的解析，是不是本地有什么网络特殊配置

Dwayneten commented 4 months ago

是不是本地有什么网络特殊配置

在docker-compose.yml中给 Higress 指定 dns 为 223.5.5.5 后正常了 感谢回复 :smiling_face_with_three_hearts:

dpy013 commented 4 months ago

各位，按照上面的说明都部署好了，打开提示API not found: /这问题怎么破？ 2024-05-31 00:52:13,142 INFO Set uid to user 0 succeeded 2024-05-31 00:52:13,144 INFO supervisord started with pid 1 2024-05-31 00:52:14,147 INFO spawned: 'apiserver' with pid 8 2024-05-31 00:52:14,150 INFO spawned: 'controller' with pid 9 2024-05-31 00:52:14,153 INFO spawned: 'pilot' with pid 10 2024-05-31 00:52:14,155 INFO spawned: 'gateway' with pid 11 2024-05-31 00:52:14,162 INFO spawned: 'console' with pid 12 2024-05-31 00:52:15,186 INFO success: apiserver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2024-05-31 00:52:15,186 INFO success: controller entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2024-05-31 00:52:15,186 INFO success: pilot entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2024-05-31 00:52:15,186 INFO success: gateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2024-05-31 00:52:15,186 INFO success: console entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

johnlanni commented 4 months ago

@dpy013 是说直接访问 127.0.0.1:8080吗，得访问127.0.0.1:3000

dpy013 commented 4 months ago

是的，访问的确实是127.0.0.1:8080 用的是腾讯云的服务器搭的。 操作系统是ubuntu24.04 lts

dpy013 commented 4 months ago

删除了原来的重新部署，并且访问了127.0.0.1:3000发送问题出现以下错误：

访问密码不正确或为空，请前往登录页输入正确的访问密码，或者在设置页填入你自己的 OpenAI API Key。

{
  "error": true,
  "msg": "wrong access code"
}
johnlanni commented 4 months ago

@dpy013 注意docker-compose.yaml里nextchat容器的CODE环境变量配置

dpy013 commented 4 months ago

把通译千问的api 密钥修改到了 - OPENAI_API_KEY=unused这个参数里了，还是不能用。

johnlanni commented 4 months ago

@dpy013 跟这个没关系

image

注意下这个

dpy013 commented 4 months ago

看来不是服务端问题，是我自己的key出问题了。 我阿里云打开了nfa，会不会是nfa影响到了通译的key？

johnlanni commented 4 months ago

应该不会的

dpy013 commented 4 months ago

刚重新生成了一个key在去看看。

dpy013 commented 4 months ago

这错误越来越奇妙了！ { "error": true, "msg": "empty access code" }

dpy013 commented 4 months ago

下面是日志： error-logs.zip

johnlanni commented 4 months ago

@dpy013 你的docker-compose脱敏发一下吧

dpy013 commented 4 months ago

以下是docker-compose.yaml文件：

version: '3.9'

networks:
  higress-net:
    external: false

services:
  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0-rc.1
    environment:
      - CONFIG_TEMPLATE=ai-proxy
      - DEFAULT_AI_SERVICE=qwen
      - DASHSCOPE_API_KEY=xx-xxxxxxxxxaaaaaessddd19sa4c97d10909
    networks:
      - higress-net
    ports:
      - "8080:8080/tcp"
      - "8001:8001/tcp"
    volumes:
      - /home/ubuntu/AI:/data
    restart: always
  nextchat:
    image: yidadaa/chatgpt-next-web
    environment:
      - CODE=xxxx1234???
      - OPENAI_API_KEY=unused
      - CUSTOM_MODELS=-all,+gpt-4o
      - BASE_URL=http://higress:8080
    networks:
      - higress-net
    ports:
      - "3000:3000/tcp"
    restart: always
dpy013 commented 4 months ago

文件名称维 docker-compose.yaml 启动命令用的是 docker-compose up -d

johnlanni commented 4 months ago

CODE环境变量去掉试试，可能是 chatgpt-next-web 的新镜像的问题

johnlanni commented 4 months ago

https://github.com/alibaba/higress/issues/1023

@dpy013 可以试试lobechat

generic-code-learn commented 4 months ago

你好报错了： { "cause": { "errno": -3001, "code": "EAI_AGAIN", "syscall": "getaddrinfo", "hostname": "higress" } }

johnlanni commented 4 months ago

你好报错了： { "cause": { "errno": -3001, "code": "EAI_AGAIN", "syscall": "getaddrinfo", "hostname": "higress" } }

定位到是新版本镜像的配置问题，你重新执行一下 docker pull higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0 ，再启动试试

caoler commented 3 months ago

version: '3.9'

networks: higress-net: external: false

services: higress: image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0 environment:

提示： { "cause": { "errno": -111, "code": "ECONNREFUSED", "syscall": "connect", "address": "172.26.0.2", "port": 8088 } }

CH3CHO commented 3 months ago

- BASE_URL=http://higress:8088/这里还是8080。 @caoler

caoler commented 3 months ago

@CH3CHO 

version: '3.9'

networks:
  higress-net:
    external: false

services:
  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0
    environment:
      - CONFIG_TEMPLATE=ai-proxy
      - DEFAULT_AI_SERVICE=qwen
      - DASHSCOPE_API_KEY=sk-24dbxxxxxxxxxxxxxxxxxxxxxxxx
    networks:
      - higress-net
    ports:
      - "8088:8080/tcp"
      - "8001:8001/tcp"
    volumes:
      - /path-to-local-config-folder:/data
    restart: always
    dns:
      - 223.5.5.5
  nextchat:
    image: yidadaa/chatgpt-next-web
    environment:
      - OPENAI_API_KEY=unused
      - CUSTOM_MODELS=-all,+gpt-4o
      - BASE_URL=http://higress:8080
    networks:
      - higress-net
    ports:
      - "3000:3000/tcp"
    restart: always
{
  "cause": {
    "errno": -111,
    "code": "ECONNREFUSED",
    "syscall": "connect",
    "address": "172.26.0.3",
    "port": 8080
  }
}
CH3CHO commented 3 months ago

@caoler

强制 docker pull higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0 一下试试。前几天有更新过这个镜像。

如果还不行的话，请提供 higress 容器中 /var/log/higress 目录下的各个日志文件以供分析。

caoler commented 3 months ago
image

@CH3CHO 这里打开之后有变化：

访问密码不正确或为空，请前往登录页输入正确的访问密码，或者在设置页填入你自己的 OpenAI API Key。


{
  "code": "InvalidApiKey",
  "message": "Invalid API-key provided.",
  "request_id": "f5a45a58-c3dc-9612-a5b6-94c1293d2408"
}
CH3CHO commented 3 months ago 
{
  "code": "InvalidApiKey",
  "message": "Invalid API-key provided.",
  "request_id": "f5a45a58-c3dc-9612-a5b6-94c1293d2408"
}

这个错误是在哪儿弹出来的？你的 sk 确定是对的吗？

caoler commented 3 months ago 
{
  "code": "InvalidApiKey",
  "message": "Invalid API-key provided.",
  "request_id": "f5a45a58-c3dc-9612-a5b6-94c1293d2408"
}

这个错误是在哪儿弹出来的？你的 sk 确定是对的吗？

对话框弹出来的

下面是一次对话请求的日志： ==> gateway.log <== {"authority":"higress:8080","bytes_received":"815","bytes_sent":"114","downstream_local_address":"172.26.0.3:8080","downstream_remote_address":"172.26.0.2:55866","duration":"211","istio_policy_status":"-","method":"POST","path":"/v1/chat/completions?path=v1&path=chat&path=completions","protocol":"HTTP/1.1","request_id":"80bca58a-6836-45ff-ae43-b6ce122e7150","requested_server_name":"-","response_code":"401","response_flags":"-","route_name":"qwen","start_time":"2024-06-06T09:47:00.834Z","trace_id":"eaca8aacef497ead8c3a417a23a8e909","upstream_cluster":"outbound|443||qwen.dns","upstream_host":"47.93.243.29:443","upstream_local_address":"172.26.0.3:43122","upstream_service_time":"208","upstream_transport_failure_reason":"-","user_agent":"Next.js Middleware","x_forwarded_for":"172.26.0.2"} {"authority":"higress:8080","bytes_received":"1131","bytes_sent":"114","downstream_local_address":"172.26.0.3:8080","downstream_remote_address":"172.26.0.2:55866","duration":"56","istio_policy_status":"-","method":"POST","path":"/v1/chat/completions?path=v1&path=chat&path=completions","protocol":"HTTP/1.1","request_id":"24f750e9-197f-48ba-8689-808f32d30a75","requested_server_name":"-","response_code":"401","response_flags":"-","route_name":"qwen","start_time":"2024-06-06T09:47:01.206Z","trace_id":"136ebab872eacaef3955a1499214466a","upstream_cluster":"outbound|443||qwen.dns","upstream_host":"47.93.243.29:443","upstream_local_address":"172.26.0.3:43122","upstream_service_time":"55","upstream_transport_failure_reason":"-","user_agent":"Next.js Middleware","x_forwarded_for":"172.26.0.2"}

==> apiserver.log <== I0606 09:47:03.166345 38 file_rest.go:262] [configmaps] higress-gateway-status-leader got I0606 09:47:03.168543 38 file_rest.go:262] [configmaps] higress-gateway-status-leader got I0606 09:47:03.169571 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-gateway-status-leader count(watcher)=1 I0606 09:47:03.215653 38 file_rest.go:262] [configmaps] higress-namespace-controller-election got I0606 09:47:03.217942 38 file_rest.go:262] [configmaps] higress-namespace-controller-election got I0606 09:47:03.218895 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-namespace-controller-election count(watcher)=1 I0606 09:47:03.339107 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-gateway-status-leader count(watcher)=1 I0606 09:47:03.339532 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-namespace-controller-election count(watcher)=1 I0606 09:47:09.639817 38 file_rest.go:301] [nodes] list count=0 I0606 09:47:10.672550 38 file_rest.go:262] [configmaps] higress-gateway-status-leader got I0606 09:47:10.675625 38 file_rest.go:262] [configmaps] higress-gateway-status-leader got I0606 09:47:10.677305 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-gateway-status-leader count(watcher)=1 I0606 09:47:10.722821 38 file_rest.go:262] [configmaps] higress-namespace-controller-election got I0606 09:47:10.725551 38 file_rest.go:262] [configmaps] higress-namespace-controller-election got I0606 09:47:10.726962 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-namespace-controller-election count(watcher)=1 I0606 09:47:10.838946 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-gateway-status-leader count(watcher)=1 I0606 09:47:10.840072 38 file_rest.go:229] event MODIFIED &TypeMeta{Kind:ConfigMap,APIVersion:v1,} higress-system/higress-namespace-controller-election count(watcher)=1 I0606 09:47:12.425643 38 file_rest.go:301] [secrets] list count=1 I0606 09:47:12.609491 38 file_rest.go:301] [secrets] list count=1

==> pilot.log <== 2024-06-06T09:47:12.710983Z info ads Push debounce stable[12] 1 for config Secret/higress-system/higress-console: 100.244251ms since last change, 100.24389ms since last push, full=false 2024-06-06T09:47:12.711063Z info ads XDS: Incremental Pushing:2024-06-06T09:46:29Z/5 ConnectedEndpoints:1 Version:2024-06-06T09:46:29Z/5

CH3CHO commented 3 months ago

@caoler

哪儿的对话框，NextChat 网页吗？看上去这个错误是通义千问返回的。建议检查一下 API Key 是否正确。

caoler commented 3 months ago

@caoler

哪儿的对话框，NextChat 网页吗？看上去这个错误是通义千问返回的。建议检查一下 API Key 是否正确。

是NextChat 网页返回的，API Key是刚刚申请的，我换一个试下