修改chatollama的chat中的参数，无法启用rerank

[heimy2000]

使用if ((process.env.COHERE_API_KEY || process.env.COHERE_BASE_URL) && process.env.COHERE_MODEL) { const options = { apiKey: process.env.COHERE_API_KEY, baseUrl: "http://localhost:1230/v1", model: "ms-marco-MiniLM-L-6-v2", topN: 4 } console.log("Cohere Rerank Options: ", options) const cohereRerank = new CohereRerank(options) rerankedDocuments = await cohereRerank.compressDocuments(relevant_docs, reformulatedQuery) console.log("Cohere reranked documents: ", rerankedDocuments) } 无法正常启用chatollama中的rerank

[heimy2000]

补充细节，部署完毕，我改了docker文件是1230：8000 回复说{"detail":"Not Found"}

[heimy2000]

补充，好像发现了问题(base) root@heimy-MS-7D70:/home/ollama/peanut-shell-main# curl --location 'http://localhost:1230/v1/rerank/' \ --header 'Content-Type: application/json' \ --data '{ "model": "ms-marco-MiniLM-L-6-v2", "query": "What is the capital of the United States?", "top_n": 3, "documents": [ "Carson City is the capital city of the American state of Nevada.", "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.", "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district." ] }' {"detail":"Method Not Allowed"}curl: (3) URL using bad/illegal format or missing URL curl: (3) URL using bad/illegal format or missing URL curl: (3) URL using bad/illegal format or missing URL curl: (3) nested brace in URL position 117: { "model": "ms-marco-MiniLM-L-6-v2", "query": "What is the capital of the United States?", "top_n": 3, "documents": [ "Carson City is the capital city of the American state of Nevada.", "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.", "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district." ] }

[heimy2000]

重新试了一次，还是不行

[sugarforever]

参考最新的 https://github.com/sugarforever/peanut-shell/blob/main/docker-compose.yaml，给HF_ENDPOINT添加了默认值

你可以把容器以非daemon运行 docker compose up

再调用一下 /v1/models 看看API响应是什么

[sugarforever]

最新的镜像解决了使用本地模型文件的问题。参考 https://github.com/sugarforever/peanut-shell/issues/5

推荐拉取最新代码或者镜像使用。

[heimy2000]

参考最新的 https://github.com/sugarforever/peanut-shell/blob/main/docker-compose.yaml，给HF_ENDPOINT添加了默认值

你可以把容器以非daemon运行docker compose up

再调用一下 /v1/models 看看API响应是什么

现在的问题成为网络失败，代理后也是失败

[heimy2000]

参考最新的 https://github.com/sugarforever/peanut-shell/blob/main/docker-compose.yaml，给HF_ENDPOINT添加了默认值

你可以把容器以非daemon运行docker compose up

再调用一下 /v1/models 看看API响应是什么

chatollama的docker版本也出现了花生壳同样的问题

[heimy2000]

研究出来一个解决方法 version: '3.1' services: peanutshell: image: ghcr.io/sugarforever/peanut-shell:latest pull_policy: always environment:

• HF_ENDPOINT=${HF_ENDPOINT:-https://hf-mirror.com} ports:
• "8000:8000" volumes:
• hf_data:/root/.cache

volumes: hf_data: 这个是docker的文件，我修改了 environment，让他适配了国内网络问题

[sugarforever]

研究出来一个解决方法 version: '3.1' services: peanutshell: image: ghcr.io/sugarforever/peanut-shell:latest pull_policy: always environment: - HF_ENDPOINT=${HF_ENDPOINT:-https://hf-mirror.com} ports: - "8000:8000" volumes: - hf_data:/root/.cache

volumes: hf_data: 这个是docker的文件，我修改了 environment，让他适配了国内网络问题

https://github.com/sugarforever/peanut-shell/blob/main/docker-compose.yaml

version: '3.1'
services:
  peanutshell:
    image: ghcr.io/sugarforever/peanut-shell:latest
    pull_policy: always
    environment:
      - HF_ENDPOINT=${HF_ENDPOINT:-https://huggingface.co}
    ports:
      - "8000:8000"
    volumes:
      - hf_data:/root/.cache

volumes:
  hf_data:

这是最新的docker compose yaml。应该支持环境变量HF_ENDPOINT了。你在同级目录下添加.env，指定HF_ENDPOINT=https://hf-mirror.com

这样应该就能使用国内的镜像 hf-mirror 了。

你可以试试吗？

[heimy2000]

研究出来一个解决方法 version: '3.1' services: peanutshell: image: ghcr.io/sugarforever/peanut-shell:latest pull_policy: always environment: - HF_ENDPOINT=${HF_ENDPOINT:-https://hf-mirror.com} ports: - "8000:8000" volumes: - hf_data:/root/.cache volumes: hf_data: 这个是docker的文件，我修改了 environment，让他适配了国内网络问题

https://github.com/sugarforever/peanut-shell/blob/main/docker-compose.yaml

version: '3.1'
services:
  peanutshell:
    image: ghcr.io/sugarforever/peanut-shell:latest
    pull_policy: always
    environment:
      - HF_ENDPOINT=${HF_ENDPOINT:-https://huggingface.co}
    ports:
      - "8000:8000"
    volumes:
      - hf_data:/root/.cache

volumes:
  hf_data:

这是最新的docker compose yaml。应该支持环境变量HF_ENDPOINT了。你在同级目录下添加.env，指定HF_ENDPOINT=https://hf-mirror.com

这样应该就能使用国内的镜像 hf-mirror 了。

你可以试试吗？

可以。目前在尝试适配chatollama，好像还是有问题

[heimy2000]

研究出来一个解决方法 version: '3.1' services: peanutshell: image: ghcr.io/sugarforever/peanut-shell:latest pull_policy: always environment: - HF_ENDPOINT=${HF_ENDPOINT:-https://hf-mirror.com} ports: - "8000:8000" volumes: - hf_data:/root/.cache volumes: hf_data: 这个是docker的文件，我修改了 environment，让他适配了国内网络问题

https://github.com/sugarforever/peanut-shell/blob/main/docker-compose.yaml

version: '3.1'
services:
  peanutshell:
    image: ghcr.io/sugarforever/peanut-shell:latest
    pull_policy: always
    environment:
      - HF_ENDPOINT=${HF_ENDPOINT:-https://huggingface.co}
    ports:
      - "8000:8000"
    volumes:
      - hf_data:/root/.cache

volumes:
  hf_data:

这是最新的docker compose yaml。应该支持环境变量HF_ENDPOINT了。你在同级目录下添加.env，指定HF_ENDPOINT=https://hf-mirror.com

这样应该就能使用国内的镜像 hf-mirror 了。

你可以试试吗？

UP大佬，请问我能否出一期视频，讲一下ubuntu部署踩得坑

[sugarforever]

研究出来一个解决方法 version: '3.1' services: peanutshell: image: ghcr.io/sugarforever/peanut-shell:latest pull_policy: always environment: - HF_ENDPOINT=${HF_ENDPOINT:-https://hf-mirror.com} ports: - "8000:8000" volumes: - hf_data:/root/.cache volumes: hf_data: 这个是docker的文件，我修改了 environment，让他适配了国内网络问题

https://github.com/sugarforever/peanut-shell/blob/main/docker-compose.yaml

version: '3.1'
services:
  peanutshell:
    image: ghcr.io/sugarforever/peanut-shell:latest
    pull_policy: always
    environment:
      - HF_ENDPOINT=${HF_ENDPOINT:-https://huggingface.co}
    ports:
      - "8000:8000"
    volumes:
      - hf_data:/root/.cache

volumes:
  hf_data:

这是最新的docker compose yaml。应该支持环境变量HF_ENDPOINT了。你在同级目录下添加.env，指定HF_ENDPOINT=https://hf-mirror.com 这样应该就能使用国内的镜像 hf-mirror 了。 你可以试试吗？

UP大佬，请问我能否出一期视频，讲一下ubuntu部署踩得坑

可以可以