janhq / cortex.cpp

Run and customize Local LLMs.
https://cortex.so
Apache License 2.0
1.94k stars 108 forks source link

bug: Incorrect content-type in chat completions response #465

Closed dpkirchner closed 3 months ago

dpkirchner commented 6 months ago

Describe the bug The content-type header returned on the /v1/chat/completions route is text/html; charset=utf-8 instead of application/json, which causes errors when using clients that use the header to decode responses.

To Reproduce Steps to reproduce the behavior:

  1. Load a model into nitro, any model will do
  2. Hit the route with curl: curl -vv -s http://localhost:3928/v1/chat/completions -H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"Whatever"}]}'
  3. See the incorrect content-type

Expected behavior The content-type header in the http response should be application/json.

Additional context

curl -vv -s http://localhost:3928/v1/chat/completions -H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"Whatever"}]}'

output:


*   Trying 127.0.0.1:3928...
* Connected to localhost (127.0.0.1) port 3928 (#0)
> POST /v1/chat/completions HTTP/1.1
> Host: localhost:3928
> User-Agent: curl/7.74.0
> Accept: */*
> content-type: application/json
> Content-Length: 51
>
* upload completely sent off: 51 out of 51 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 309
< content-type: text/html; charset=utf-8
< server: drogon/1.9.2
< access-control-allow-origin: *
< date: Wed, 20 Mar 2024 22:58:27 GMT
<
* Connection #0 to host localhost left intact
{"choices":[{"finish_reason":null,"index":0,"message": [snipped]```
louis-jan commented 3 months ago

It's addressed. Could you please try again @dpkirchner?

dpkirchner commented 3 months ago

lgtm, thanks!