search

nextcloud / integration_openai

OpenAI integration in Nextcloud
GNU Affero General Public License v3.0
48 stars 11 forks source link

[Bug]: OpenAI error: Client error: `POST http://192.168.73.78:8000/v1/chat/completions` resulted in a `404 Not Found` response: File Not Found #44

Open Stargate256 opened 11 months ago

Stargate256 commented 11 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

I am trying to use the local text generation integration with llama.cpp

When I try to generate text I get: OpenAI error: Client error: POST http://192.168.73.78:8000/v1/chat/completions resulted in a 404 Not Found response: File Not Found

The problem is that llama.cpp uses: "http://192.168.73.78:8000//completion"

Steps to reproduce

  1. Download and configure llama.cpp, run with: "./server -m /mnt/models/llama-2-13b-chat-limarp-v2-merged.ggmlv3.Q4_0.guff -ngl 512 -c 2048 --port 8000 --host 0.0.0.0"
  2. Download, install and configure the OpenAI and Local AI integration, setting local address to the address of a machine running llama.cpp
  3. Prompt the model.

Expected behavior

It should generate text.

Installation method

Community Manual installation with Archive

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "27.0.2.1",
        "overwrite.cli.url": "https:***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "app_install_overwrite": [
            "documents",
            "hancomoffice"
        ],
        "maintenance": false,
        "mail_smtpdebug": true,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpport": "465",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "app.mail.verify-tls-peer": false,
        "allow_local_remote_servers": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "theme": "",
        "loglevel": 2
    }
}

List of activated Apps

No response

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

julien-nc commented 11 months ago

Hi, thanks for the detailed report.

What do you do on the Nextcloud side? Are you using the "text generation" smart picker provider? If so it's weird that a request to /v1/chat/completions is done by NC. This should only happen if the model name starts with gpt-. In your case it should make a request to /v1/completions.

Which project exactly are you using to create the web service? LocalAI can use llama.cpp via /v1/chat/completions (https://localai.io/model-compatibility/llama-cpp/) so I assume you're using something else.

I don't have a good internet connection right now, I'll try to play with LocalAI and llama models next week. If we can identify clear cases the integration app could be adjusted to request different endpoints depending on the selected model.

Stargate256 commented 11 months ago

Are you using the "text generation" smart picker provider? - yes

Which project exactly are you using to create the web service? - llama.cpp server with

ynott commented 10 months ago

I am facing the same problem.

I have LocalAI running in a virtual machine separate from the Nextcloud server.

0. Environment

Nextcloud version:"27.1.2" OpenAI and LocalAI integration: 1.0.13 LocalAI: v1.30.0

1. LocalAI condition

$ docker ps
CONTAINER ID   IMAGE                               COMMAND                   CREATED          STATUS                    PORTS      NAMES
3b789d3b04d1   quay.io/go-skynet/local-ai:latest   "/build/entrypoint.s…"   21 minutes ago   Up 20 minutes (healthy)   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   localai-api-1

From the Nextcloud machine, the curl command in the CLI confirmed that LocalAI was working.

Model information is also obtained as shown in the following screenshot. image

2. The setting uses LocalAI, but /v1/completions is requested.

2-1. Text generation with Nextcloud Talk

image

2. Nextcloud Logs

Nextcloud Logs
``` { "reqId": "z51rHBXxG5K8BHBsutqW", "level": 2, "time": "2023-10-17 15:28:59", "remoteAddr": "172.16.99.200", "user": "ncadmin", "app": "integration_openai", "method": "POST", "url": "/apps/integration_openai/completions", "message": "OpenAI API error : Server error: `POST http://172.16.99.144:8080/v1/completions` resulted in a `500 Internal Server Error` response:\n{\"error\":{\"code\":500,\"message\":\"rpc error: code = Unknown desc = unimplemented\",\"type\":\"\"}}\n", "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36", "version": "27.1.2.1", "exception": { "Exception": "GuzzleHttp\\Exception\\ServerException", "Message": "Server error: `POST http://172.16.99.144:8080/v1/completions` resulted in a `500 Internal Server Error` response:\n{\"error\":{\"code\":500,\"message\":\"rpc error: code = Unknown desc = unimplemented\",\"type\":\"\"}}\n", "Code": 500, "Trace": [ { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php", "line": 69, "function": "create", "class": "GuzzleHttp\\Exception\\RequestException", "type": "::", "args": [ "*** sensitive parameters replaced ***" ] }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php", "line": 204, "function": "GuzzleHttp\\{closure}", "class": "GuzzleHttp\\Middleware", "type": "::", "args": [ "*** sensitive parameters replaced ***" ] }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php", "line": 153, "function": "callHandler", "class": "GuzzleHttp\\Promise\\Promise", "type": "::" }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/TaskQueue.php", "line": 48, "function": "GuzzleHttp\\Promise\\{closure}", "class": "GuzzleHttp\\Promise\\Promise", "type": "::", "args": [ "*** sensitive parameters replaced ***" ] }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php", "line": 248, "function": "run", "class": "GuzzleHttp\\Promise\\TaskQueue", "type": "->" }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php", "line": 224, "function": "invokeWaitFn", "class": "GuzzleHttp\\Promise\\Promise", "type": "->" }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php", "line": 269, "function": "waitIfPending", "class": "GuzzleHttp\\Promise\\Promise", "type": "->" }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php", "line": 226, "function": "invokeWaitList", "class": "GuzzleHttp\\Promise\\Promise", "type": "->" }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php", "line": 62, "function": "waitIfPending", "class": "GuzzleHttp\\Promise\\Promise", "type": "->" }, { "file": "/var/www/html/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php", "line": 187, "function": "wait", "class": "GuzzleHttp\\Promise\\Promise", "type": "->" }, { "file": "/var/www/html/nextcloud/lib/private/Http/Client/Client.php", "line": 301, "function": "request", "class": "GuzzleHttp\\Client", "type": "->" }, { "file": "/var/www/html/nextcloud/apps/integration_openai/lib/Service/OpenAiAPIService.php", "line": 357, "function": "post", "class": "OC\\Http\\Client\\Client", "type": "->" }, { "file": "/var/www/html/nextcloud/apps/integration_openai/lib/Service/OpenAiAPIService.php", "line": 112, "function": "request", "class": "OCA\\OpenAi\\Service\\OpenAiAPIService", "type": "->" }, { "file": "/var/www/html/nextcloud/apps/integration_openai/lib/Controller/OpenAiAPIController.php", "line": 85, "function": "createCompletion", "class": "OCA\\OpenAi\\Service\\OpenAiAPIService", "type": "->", "args": [ "*** sensitive parameters replaced ***" ] }, { "file": "/var/www/html/nextcloud/lib/private/AppFramework/Http/Dispatcher.php", "line": 230, "function": "createCompletion", "class": "OCA\\OpenAi\\Controller\\OpenAiAPIController", "type": "->", "args": [ "*** sensitive parameters replaced ***" ] }, { "file": "/var/www/html/nextcloud/lib/private/AppFramework/Http/Dispatcher.php", "line": 137, "function": "executeController", "class": "OC\\AppFramework\\Http\\Dispatcher", "type": "->" }, { "file": "/var/www/html/nextcloud/lib/private/AppFramework/App.php", "line": 183, "function": "dispatch", "class": "OC\\AppFramework\\Http\\Dispatcher", "type": "->" }, { "file": "/var/www/html/nextcloud/lib/private/Route/Router.php", "line": 315, "function": "main", "class": "OC\\AppFramework\\App", "type": "::" }, { "file": "/var/www/html/nextcloud/lib/base.php", "line": 1068, "function": "match", "class": "OC\\Route\\Router", "type": "->" }, { "file": "/var/www/html/nextcloud/index.php", "line": 36, "function": "handleRequest", "class": "OC", "type": "::" } ], "File": "/var/www/html/nextcloud/3rdparty/guzzlehttp/guzzle/src/Exception/RequestException.php", "Line": 113, "message": "OpenAI API error : Server error: `POST http://172.16.99.144:8080/v1/completions` resulted in a `500 Internal Server Error` response:\n{\"error\":{\"code\":500,\"message\":\"rpc error: code = Unknown desc = unimplemented\",\"type\":\"\"}}\n", "response_body": {}, "exception": {}, "CustomMessage": "OpenAI API error : Server error: `POST http://172.16.99.144:8080/v1/completions` resulted in a `500 Internal Server Error` response:\n{\"error\":{\"code\":500,\"message\":\"rpc error: code = Unknown desc = unimplemented\",\"type\":\"\"}}\n" } } ```

2-2. LocalAI Logs

This is the log output from the LocalAI side with Debug.

image

ynott commented 10 months ago

@julien-nc

This is like you are saying. https://github.com/nextcloud/integration_openai/issues/44#issuecomment-1719096751 This is due to the fact that /completions is called if the model name does not start with 'gpt-'.

I am using a non-'gpt-' model in LocalAI, so it is /completions.

But, since /chat/completions is already available in LocalAI, this condition is no longer necessary.

📖 Text generation (GPT) :: LocalAI documentation https://localai.io/features/text-generation/

This test query works fine.

curl http://172.16.xxx.yyy:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
      "model": "luna-ai-llama2",
      "messages": [{"role": "user", "content": "How are you?"}],
      "temperature": 0.9
    }'
MB-Finski commented 10 months ago

Ok, I think we'll set the default endpoint to /chat/completions and add a setting to specify the endpoint manually just in case some is still using an old version of LocalAI.

liucoj commented 5 months ago

Hi, i have a same issue but on NC hosted on Hetzner server. I run an API AI via Ollama on a public url (es: https://something1234.com) when I try to add in LocalAI Url i notice a "404 not found /v1/models" on AI server log. But in main interface of this plugin I haven't a model menu to choose from (as Ynott user posted before). Any idea?

Stargate256 commented 5 months ago

I switched from llamacpp to koboldcpp(https://github.com/LostRuins/koboldcpp) and the now the nextcloud LocalAI integration works.

LukeVassallo commented 2 months ago

Is there a temporary workaround for this?

meonkeys commented 1 week ago

Is there a temporary workaround for this?

I mean, maybe koboldcpp? I'm not having any luck with that one, personally.

open-webui isn't working for me either, but my errors look a bit different than described above: https://help.nextcloud.com/t/ollama-integration-with-nexcloud/180302/6

LocalAI does work for me, FWIW. Have you tried that? Their "AIO" images are super handy: https://localai.io/basics/container/