Support for GPT-4 via API

[bulletproofmedic]

I would like to use this chat-ui for communication with GPT4 as its much nicer than the one I threw together. I have searched for info on setting this up, but didn't find anything.

I think I'll have time on Sunday to attempt to get it working in my fork, but if anyone feels like getting to it before me that would be wonderful.

[gururise]

I think one way would be to write an endpoint that emulates the /generate_stream endpoint in the text-generation-server. and just returns the response from OpenAI GPT-4.

Doing it that way, you wouldn't have to make any changes to the front-end.

[gururise]

I wrote a FastAPI endpoint that integrates nicely with OpenAI and HuggingChat UI.

Check it out here: https://github.com/gururise/openai_text_generation_inference_server

[julien-c]

That's great @gururise! If you're interested (and would be comfortable writing some Typescript) we'd welcome this code directly in chat-ui

[aniket-synth]

Hi, Used the https://github.com/gururise/openai_text_generation_inference_server with chat-ui and its working fine, the response gets displayed in the UI but the first message of every new conversation results in this error:

Error: {"detail":[{"type":"missing","loc":["body","parameters"],"msg":"Field required","input":{"temperature":0.9,"truncate":1000,"max_new_tokens":50,"return_full_text":false,"inputs":" User: hi Assistant: "},"url":"https://errors.pydantic.dev/2.3/v/missing"}]} at Module.generateFromDefaultEndpoint (/home/chat-ui/src/lib/server/generateFromDefaultEndpoint.ts:58:11) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async POST (/home/chat-ui/src/routes/conversation/[id]/summarize/+server.ts:32:26) at async Module.render_endpoint (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:47:20) at async resolve (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:388:17) at async Object.handle (/home/chat-ui/src/hooks.server.ts:66:20) at async Module.respond (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:259:20) at async file:///home/chat-ui/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:506:22

[julien-blanchon]

This PR (https://github.com/huggingface/chat-ui/pull/443) add the OpenAI API by implementing a new route directly on the chat-ui server. It's the same idea as the @gururise python script but host on the server.

This could still be a little bit buggy, don't hesitate to try so I can have feedback

[aniket-synth]

Hi, Used the https://github.com/gururise/openai_text_generation_inference_server with chat-ui and its working fine, the response gets displayed in the UI but the first message of every new conversation results in this error:

Error: {"detail":[{"type":"missing","loc":["body","parameters"],"msg":"Field required","input":{"temperature":0.9,"truncate":1000,"max_new_tokens":50,"return_full_text":false,"inputs":" User: hi Assistant: "},"url":"https://errors.pydantic.dev/2.3/v/missing"}]} at Module.generateFromDefaultEndpoint (/home/chat-ui/src/lib/server/generateFromDefaultEndpoint.ts:58:11) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async POST (/home/chat-ui/src/routes/conversation/[id]/summarize/+server.ts:32:26) at async Module.render_endpoint (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:47:20) at async resolve (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:388:17) at async Object.handle (/home/chat-ui/src/hooks.server.ts:66:20) at async Module.respond (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:259:20) at async file:///home/chat-ui/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:506:22

Was earlier getting this error, now on the same code I get this: SyntaxError: Unexpected token d in JSON at position 0 at JSON.parse () at Module.generateFromDefaultEndpoint (/home/aniket/Synth/llama-model/chat-ui/src/lib/server/generateFromDefaultEndpoint.ts:73:30) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async POST (/home/aniket/Synth/llama-model/chat-ui/src/routes/conversation/[id]/summarize/+server.ts:50:26) at async Module.render_endpoint (/home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:47:20) at async resolve (/home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:388:17) at async Object.handle (/home/aniket/Synth/llama-model/chat-ui/src/hooks.server.ts:66:20) at async Module.respond (/home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:259:20) at async file:///home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:506:22

[julien-blanchon]

@aniket-synth You could try to install the chat-ui version from my fork repo (https://github.com/julien-blanchon/chat-ui/tree/add-chatgpt-api) on the add-chatgpt-api branch, this sould work well. I did a PR to add this to chat-ui.

Here is a short documentation about addding OpenAI model using your env.local:

---

You can also specify your OpenAI API key as an endpoint for chat-ui. The config goes like this:

  "endpoints": [{
    "host": "openai",
    "apiKey": "sk-...",
    "model": "gpt-3.5-turbo",
    "temperature": 0.9,
    "max_tokens": 100
  }],

Note that with the "openai" host the following parameters are required:

  "userMessageToken": "<|user|>",
  "assistantMessageToken": "<|assistant|>",
  "userMessageEndToken": "</s>",
  "assistantMessageEndToken": "</s>",
  "preprompt": "<|system|>You are a helpful assistant...</s>",

[aniket-synth]

Hi, Used the https://github.com/gururise/openai_text_generation_inference_server with chat-ui and its working fine, the response gets displayed in the UI but the first message of every new conversation results in this error: Error: {"detail":[{"type":"missing","loc":["body","parameters"],"msg":"Field required","input":{"temperature":0.9,"truncate":1000,"max_new_tokens":50,"return_full_text":false,"inputs":" User: hi Assistant: "},"url":"https://errors.pydantic.dev/2.3/v/missing"}]} at Module.generateFromDefaultEndpoint (/home/chat-ui/src/lib/server/generateFromDefaultEndpoint.ts:58:11) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async POST (/home/chat-ui/src/routes/conversation/[id]/summarize/+server.ts:32:26) at async Module.render_endpoint (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:47:20) at async resolve (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:388:17) at async Object.handle (/home/chat-ui/src/hooks.server.ts:66:20) at async Module.respond (/home/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:259:20) at async file:///home/chat-ui/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:506:22

Was earlier getting this error, now on the same code I get this: SyntaxError: Unexpected token d in JSON at position 0 at JSON.parse () at Module.generateFromDefaultEndpoint (/home/aniket/Synth/llama-model/chat-ui/src/lib/server/generateFromDefaultEndpoint.ts:73:30) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async POST (/home/aniket/Synth/llama-model/chat-ui/src/routes/conversation/[id]/summarize/+server.ts:50:26) at async Module.render_endpoint (/home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:47:20) at async resolve (/home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:388:17) at async Object.handle (/home/aniket/Synth/llama-model/chat-ui/src/hooks.server.ts:66:20) at async Module.respond (/home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/runtime/server/respond.js:259:20) at async file:///home/aniket/Synth/llama-model/chat-ui/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:506:22

@gururise Upon investigating, realised this error is received during the 'summarize' feature. How to resolve this?

[steventran1983]

Does anyway we build the API and just respone the string or json without stream ? example I build the API and response like { "content":"Here is the sample" }

and I would like the FrontEnd ChatUI show that message. Thank yo

[steventran1983]

Can someone help or suggest build the API [gururise] but not query data from ChatGPT AIP , just return a content of string that Chat-UI can accept . @gururise , could you please help to guide me