search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference
https://localai.io
MIT License
25.73k stars 1.93k forks source link

Hello, cant connect with NextCloud #3608

Open userbox020 opened 1 month ago

userbox020 commented 1 month ago

I have setting up LocalAI working fine, and NextCloud even detect the models list of LocalAI but when I use any of the features to talk with the LLM it doesnt fetch anything, it get the follow error

GET /apps/assistant/chat/check_generation?taskId=20&sessionId=21 HTTP/1.1
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Cookie: oc_sessionPassphrase=bgHjp8Tyr9s29ZBaTrH7iPAFHR1qe67PWMhdXBo9DKozrPQgxWP2lZyVfWrmlaIuyLddLrkLtacrstoEGQfwx55Sd%2BouuDeo5JZFoeezhAQpd6rLG4tFPCbiDbCQkMRy; nc_sameSiteCookielax=true; nc_sameSiteCookiestrict=true; ocj2thp9x9dg=a39aba2bb49eec84a8c772bb1b874c6f; nc_username=admin; nc_token=5H%2FNfg4NPPo%2BtewCKTl4%2FAtTRQQRviM9; nc_session_id=a39aba2bb49eec84a8c772bb1b874c6f
Host: localhost:8070
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36
X-Requested-With: XMLHttpRequest, XMLHttpRequest
requesttoken: OUgglgKesBwaxlFwTA2Fq4dbST6P5ssrktPuoLYErE4=:SQNNo3DJ6SR9nDQfCUPun98sE2n+sK9Pxb6jxsB1gw0=
sec-ch-ua: "Chromium";v="128", "Not;A=Brand";v="24", "Google Chrome";v="128"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"

HTTP/1.1 417 Expectation Failed
Date: Fri, 20 Sep 2024 09:07:03 GMT
Server: Apache/2.4.62 (Debian)
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: noindex, nofollow
X-XSS-Protection: 1; mode=block
X-Powered-By: PHP/8.2.23
Content-Security-Policy: default-src 'none';base-uri 'none';manifest-src 'self';frame-ancestors 'none'
X-Request-Id: 1CXMQ5oeDFUkY6qEm2b4
Cache-Control: no-cache, no-store, must-revalidate
Feature-Policy: autoplay 'none';camera 'none';fullscreen 'none';geolocation 'none';microphone 'none';payment 'none'
Content-Length: 17
Keep-Alive: timeout=5, max=66
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8

{"task_status":1}
userbox020 commented 1 month ago

image

userbox020 commented 1 month ago

image

rabner commented 1 month ago

It seems I have the same issue since first started setting up LocalAI AIO docker on CPU few weeks ago. So far I've managed to narrow down the issue to 504 Gateway Timeout - nextcloud log:

{
    "reqId": "Zu2K5dJu08_G0CoBPxYoHwAAAAQ",
    "level": 3,
    "time": "2024-09-20T14:48:31+00:00",
    "remoteAddr": "192.168.x.xx",
    "app": "index",
    "method": "GET",
    "url": "/apps/assistant/chat/generate?sessionId=15",
    "message":"{\"Exception\":\"OCP\\\\TextProcessing\\\\Exception\\\\TaskFailureException\",\"Message\":\"LanguageModel call using provider LocalAI failed: OpenA
I/LocalAI request failed: API request error: Server error: `POST http: //localai.lan/v1/completions` resulted in a `504 Gateway Time-out` response:\\n<html>\\r\\n<head><title>504 Gateway Time-out</title></h
ead>\\r\\n<body>\\r\\n<center><h1>504 Gateway Time-out</h1></center>\\r\\n<hr><c (truncated...)\\n\",\"Code\":0,\"Trace\":[{\"file\":\"/var/www/html/nextcloud/apps/assistant/lib/Controller/ChattyLLMControl
ler.php\",\"line\":444,\"function\":\"runTask\",\"class\":\"OC\\\\TextProcessing\\\\Manager\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/apps/assistant/lib/Controller/ChattyLLMController.php\",\"
line\":278,\"function\":\"queryLLM\",\"class\":\"OCA\\\\Assistant\\\\Controller\\\\ChattyLLMController\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/lib/private/AppFramework/Http/Dispatcher.php\",
\"line\":232,\"function\":\"generateForSession\",\"class\":\"OCA\\\\Assistant\\\\Controller\\\\ChattyLLMController\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/lib/private/AppFramework/Http/Dispa
tcher.php\",\"line\":138,\"function\":\"executeController\",\"class\":\"OC\\\\AppFramework\\\\Http\\\\Dispatcher\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/lib/private/AppFramework/App.php\",\"
line\":184,\"function\":\"dispatch\",\"class\":\"OC\\\\AppFramework\\\\Http\\\\Dispatcher\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/lib/private/Route/Router.php\",\"line\":331,\"function\":\"m
ain\",\"class\":\"OC\\\\AppFramework\\\\App\",\"type\":\"::\"},{\"file\":\"/var/www/html/nextcloud/lib/base.php\",\"line\":1058,\"function\":\"match\",\"class\":\"OC\\\\Route\\\\Router\",\"type\":\"->\"},{
\"file\":\"/var/www/html/nextcloud/index.php\",\"line\":49,\"function\":\"handleRequest\",\"class\":\"OC\",\"type\":\"::\"}],\"File\":\"/var/www/html/nextcloud/lib/private/TextProcessing/Manager.php\",\"Li
ne\":145,\"Previous\":{\"Exception\":\"RuntimeException\",\"Message\":\"OpenAI/LocalAI request failed: API request error: Server error: `POST http://localai.lan/v1/completions` resulted in a `504 Gateway T
ime-out` response:\\n<html>\\r\\n<head><title>504 Gateway Time-out</title></head>\\r\\n<body>\\r\\n<center><h1>504 Gateway Time-out</h1></center>\\r\\n<hr><c (truncated...)\\n\",\"Code\":0,\"Trace\":[{\"fi
le\":\"/var/www/html/nextcloud/lib/public/TextProcessing/Task.php\",\"line\":103,\"function\":\"process\",\"class\":\"OCA\\\\OpenAi\\\\TextProcessing\\\\FreePromptProvider\",\"type\":\"->\"},{\"file\":\"/v
ar/www/html/nextcloud/lib/private/TextProcessing/Manager.php\",\"line\":136,\"function\":\"visitProvider\",\"class\":\"OCP\\\\TextProcessing\\\\Task\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/a
pps/assistant/lib/Controller/ChattyLLMController.php\",\"line\":444,\"function\":\"runTask\",\"class\":\"OC\\\\TextProcessing\\\\Manager\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/apps/assistan
t/lib/Controller/ChattyLLMController.php\",\"line\":278,\"function\":\"queryLLM\",\"class\":\"OCA\\\\Assistant\\\\Controller\\\\ChattyLLMController\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/li
b/private/AppFramework/Http/Dispatcher.php\",\"line\":232,\"function\":\"generateForSession\",\"class\":\"OCA\\\\Assistant\\\\Controller\\\\ChattyLLMController\",\"type\":\"->\"},{\"file\":\"/var/www/html/
nextcloud/lib/private/AppFramework/Http/Dispatcher.php\",\"line\":138,\"function\":\"executeController\",\"class\":\"OC\\\\AppFramework\\\\Http\\\\Dispatcher\",\"type\":\"->\"},{\"file\":\"/var/www/html/ne
xtcloud/lib/private/AppFramework/App.php\",\"line\":184,\"function\":\"dispatch\",\"class\":\"OC\\\\AppFramework\\\\Http\\\\Dispatcher\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/lib/private/Rou
te/Router.php\",\"line\":331,\"function\":\"main\",\"class\":\"OC\\\\AppFramework\\\\App\",\"type\":\"::\"},{\"file\":\"/var/www/html/nextcloud/lib/base.php\",\"line\":1058,\"function\":\"match\",\"class\"
:\"OC\\\\Route\\\\Router\",\"type\":\"->\"},{\"file\":\"/var/www/html/nextcloud/index.php\",\"line\":49,\"function\":\"handleRequest\",\"class\":\"OC\",\"type\":\"::\"}],\"File\":\"/var/www/html/nextcloud/
apps/integration_openai/lib/TextProcessing/FreePromptProvider.php\",\"Line\":46},\"message\":\"LanguageModel call using provider LocalAI failed: OpenAI/LocalAI request failed: API request error: Server err
or: `POST http: //localai.lan/v1/completions` resulted in a `504 Gateway Time-out` response:\\n<html>\\r\\n<head><title>504 Gateway Time-out</title></head>\\r\\n<body>\\r\\n<center><h1>504 Gateway Time-out<
/h1></center>\\r\\n<hr><c (truncated...)\\n\",\"exception\":{},\"CustomMessage\":\"LanguageModel call using provider LocalAI failed: OpenAI/LocalAI request failed: API request error: Server error: `POST ht
tp: //localai.lan/v1/completions` resulted in a `504 Gateway Time-out` response:\\n<html>\\r\\n<head><title>504 Gateway Time-out</title></head>\\r\\n<body>\\r\\n<center><h1>504 Gateway Time-out</h1></center
>\\r\\n<hr><c (truncated...)\\n\"}",
    "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0",
    "version": "29.0.7.1"
}

This happens when LocalAI instance is behind Nginx Proxy Manager using local domain name. When I use IP there is no obvious error, but LocalAI docker logs shows that response is eventually generated.

I've managed to solve that by increasing timeout on Nextcloud VM in nextcloud/.user.ini by adding (I'm not sure if increasing timeouts is right way):

php_value max_input_time 3600
php_value max_execution_time 3600

source

So now text prompts work, but for some reason response is very long and AI is chatting with 'me' on it's own. I suspect it might be something with prompt in Nextcloud.

But now if I change back to domain name I'm getting 504 Gateway timeout again, so will have to dig in to Nginx config - any ideas?

userbox020 commented 1 month ago

@rabner thanks for your help bro, I increased the timeouts like you told me on the .user.ini but still getting same error

 ChattyLLMInputForm.vue:574

{
  "stack": "X@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:33123\njt@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:44544\ng@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:49220\nEventHandlerNonNull*70715/Vt</<@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:49479\n70715/Vt<@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:48808\nhe@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:55714\n_request@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:58541\nrequest@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:57068\n70715/</we.prototype[t]@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:58836\n70715/o/<@http://localhost:8080/custom_apps/assistant/js/assistant-axios-lazy.js?v=fc9be637c540e816beb7:2:27421\n22974/pollGenerationTask/</this.pollMessageGenerationTimerId<@http://localhost:8080/custom_apps/assistant/js/assistant-assistant-modal-lazy.js?v=5858c70ce6cd3b1bf029:1:75703\nsetInterval handler*22974/pollGenerationTask/<@http://localhost:8080/custom_apps/assistant/js/assistant-assistant-modal-lazy.js?v=5858c70ce6cd3b1bf029:1:75674\npollGenerationTask@http://localhost:8080/custom_apps/assistant/js/assistant-assistant-modal-lazy.js?v=5858c70ce6cd3b1bf029:1:75619\nrunGenerationTask@http://localhost:8080/custom_apps/assistant/js/assistant-assistant-modal-lazy.js?v=5858c70ce6cd3b1bf029:1:74766\nasync*newMessage@http://localhost:8080/custom_apps/assistant/js/assistant-assistant-modal-lazy.js?v=5858c70ce6cd3b1bf029:1:73843\nasync*handleSubmit@http://localhost:8080/custom_apps/assistant/js/assistant-assistant-modal-lazy.js?v=5858c70ce6cd3b1bf029:1:70457\nasync*va@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:221685\na@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:214375\nva@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:221685\n85471/e.prototype.$emit@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:243546\nsubmit@http://localhost:8080/custom_apps/assistant/js/assistant-assistant-modal-lazy.js?v=5858c70ce6cd3b1bf029:1:67271\nva@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:221685\na@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:214375\nva@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:221685\n85471/e.prototype.$emit@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:243546\nonEnter@http://localhost:8080/custom_apps/assistant/js/assistant-vendors-node_modules_nextcloud_vue_dist_Components_NcAppNavigationNew_mjs-node_modules_nextcl-28eca6.js?v=8fe271131d67e837ec65:2:567606\n42796/R/<.on.keydown<@http://localhost:8080/custom_apps/assistant/js/assistant-vendors-node_modules_nextcloud_vue_dist_Components_NcAppNavigationNew_mjs-node_modules_nextcl-28eca6.js?v=8fe271131d67e837ec65:2:571115\nva@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:221685\na@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:214454\n85471/Do/s._wrapper@http://localhost:8080/custom_apps/assistant/js/assistant-main.js?v=06ec47aa-0:2:254591\n",
  "message": "Request failed with status code 417",
  "name": "AxiosError",
  "code": "ERR_BAD_REQUEST",
  "config": {
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    },
    "adapter": [
      "xhr",
      "http",
      "fetch"
    ],
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "env": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "requesttoken": "tSslCxlJ4AS/CVIa0wZOUpo1uRefeiiG48WlInf+J0g=:mnFuOkMmrUjUXhhVvEAHBMIA+CX6CRDljbbXVAKWYhk=",
      "X-Requested-With": "XMLHttpRequest"
    },
    "params": {
      "taskId": 1,
      "sessionId": 1
    },
    "method": "get",
    "url": "/apps/assistant/chat/check_generation"
  },
  "request": {},
  "response": {
    "data": {
      "task_status": 1
    },
    "status": 417,
    "statusText": "Expectation Failed",
    "headers": {
      "cache-control": "no-cache, no-store, must-revalidate",
      "connection": "Keep-Alive",
      "content-length": "17",
      "content-security-policy": "default-src 'none';base-uri 'none';manifest-src 'self';frame-ancestors 'none'",
      "content-type": "application/json; charset=utf-8",
      "date": "Fri, 20 Sep 2024 21:04:42 GMT",
      "feature-policy": "autoplay 'none';camera 'none';fullscreen 'none';geolocation 'none';microphone 'none';payment 'none'",
      "keep-alive": "timeout=5, max=35",
      "referrer-policy": "no-referrer",
      "server": "Apache/2.4.62 (Debian)",
      "x-content-type-options": "nosniff",
      "x-frame-options": "SAMEORIGIN",
      "x-permitted-cross-domain-policies": "none",
      "x-powered-by": "PHP/8.2.23",
      "x-request-id": "uUTJzffketuPRiXStzqx",
      "x-robots-tag": "noindex, nofollow",
      "x-xss-protection": "1; mode=block"
    },
    "config": {
      "transitional": {
        "silentJSONParsing": true,
        "forcedJSONParsing": true,
        "clarifyTimeoutError": false
      },
      "adapter": [
        "xhr",
        "http",
        "fetch"
      ],
      "transformRequest": [
        null
      ],
      "transformResponse": [
        null
      ],
      "timeout": 0,
      "xsrfCookieName": "XSRF-TOKEN",
      "xsrfHeaderName": "X-XSRF-TOKEN",
      "maxContentLength": -1,
      "maxBodyLength": -1,
      "env": {},
      "headers": {
        "Accept": "application/json, text/plain, */*",
        "requesttoken": "tSslCxlJ4AS/CVIa0wZOUpo1uRefeiiG48WlInf+J0g=:mnFuOkMmrUjUXhhVvEAHBMIA+CX6CRDljbbXVAKWYhk=",
        "X-Requested-With": "XMLHttpRequest"
      },
      "params": {
        "taskId": 1,
        "sessionId": 1
      },
      "method": "get",
      "url": "/apps/assistant/chat/check_generation"
    },
    "request": {}
  },
  "status": 417
}

Its totally NextCloud problem, going to close the issue in few days just to check if someone can suggest a solution too. Thanks for your time