huggingface / chat-ui

Open source codebase powering the HuggingChat app
https://huggingface.co/chat
Apache License 2.0
7.61k stars 1.12k forks source link

Cohere endpoint type doesn't support tools #1404

Open nsarrazin opened 3 months ago

nsarrazin commented 3 months ago

Bug description

The cohere endpoint type doesn't work well with our current tools setup and errors with cannot specify both message and tool_results in multistep mode

Steps to reproduce

  1. Set a .env.local with a model using cohere endpoint type, and set tools: true on that model.
  2. Start a conversation with tool enabled

Context

Logs

 Error: {"message":"invalid request: cannot specify both message and tool_results in multistep mode"}
at eval (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/endpoints/cohere/endpointCohere.ts:93:17)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async eval (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/endpoints/cohere/endpointCohere.ts:70:18)
at async Module.generate (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/textGeneration/generate.ts:8:20)
at async textGenerationWithoutTitle (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/textGeneration/index.ts:54:3)
at async Module.mergeAsyncGenerators (C:/Users/achar/Documents/aditya/chat-ui/src/lib/utils/mergeAsyncGenerators.ts:13:34)
at async Module.textGeneration (C:/Users/achar/Documents/aditya/chat-ui/src/lib/server/textGeneration/index.ts:24:3)
at async Object.start (C:/Users/achar/Documents/aditya/chat-ui/src/routes/conversation/[id]/+server.ts:329:26) {
[cause]: BadRequestError: BadRequestError
Status code: 400
Body: {}
at CohereClient. (C:\Users\achar\Documents\aditya\chat-ui\node_modules\cohere-ai\Client.js:112:31)
at Generator.next ()
at fulfilled (C:\Users\achar\Documents\aditya\chat-ui\node_modules\cohere-ai\Client.js:31:58)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
statusCode: 400,
body: ReadableStream { locked: false, state: 'closed', supportsBYOB: false }

Config

MODELS=`[
  {
    "name": "command-r-plus",
    "tools": true,
    "displayName": "Command R+",
    "endpoints": [{
      "type": "cohere",
    }]
  }
]`
dvirmor commented 1 month ago

Hi, I'm new. I don't know if custom pull requests are welcome here. I also encounter this issues and I've found a solution: add force_single_step: true to cohere.chatStream calling in the endpointCohere.ts file.

This is my pull request: https://github.com/huggingface/chat-ui/pull/1490

I hope this is fine

nsarrazin commented 1 month ago

PRs are very welcome! thanks for looking at this @dvirmor i'll review :rocket: