search

nextcloud / context_chat_backend

GNU Affero General Public License v3.0
5 stars 5 forks source link

[bug]: Connection lost/timeout before reply #47

Closed socialize-IT closed 5 months ago

socialize-IT commented 5 months ago

Describe the bug If I send a question via the assistant a connection is made. After 30 seconds (this means BEFORE an answer) the connection is closed and I get an error from the assistant. The docker logs of the backend nervertheless show me that he action seems to be completed after that, but the answer never reaches me in nextcloud.

To Reproduce Steps to reproduce the behavior:

  1. Install all aps in the right order
  2. Ask a question
  3. Get the timeout error

Expected behavior Get an answer in the assistant.

Server logs (if applicable)

``` Error during request to ExApp (context_chat_backend): Server error: `POST https://some_domain:23000/query` resulted in a `504 Gateway Time-out` response:

504 Gateway Time-out

The server didn't respond in time. ```

Context Chat Backend logs (if applicable, from the docker container)

``` 2024-04-27T08:20:02.171029630Z TRACE: 127.0.0.1:37782 - HTTP connection made 2024-04-27T08:20:02.181210232Z TRACE: 127.0.0.1:37782 - ASGI [3634] Started scope={'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.4'}, 'http_version': '1.1', 'server': ('127.0.0.1', 23000), 'client': ('127.0.0.1', 37782), 'scheme': 'http', 'method': 'POST', 'root_path': '', 'path': '/query', 'raw_path': b'/query', 'query_string': b'', 'headers': '<...>', 'state': {}} 2024-04-27T08:20:02.185667910Z TRACE: 127.0.0.1:37782 - ASGI [3634] Receive {'type': 'http.request', 'body': '<97 bytes>', 'more_body': False} 2024-04-27T08:20:02.195724561Z query: userId='windhab_root' query='{"prompt":"How can I activate office?"}' useContext=True scopeType=None scopeList=None ctxLimit=10 2024-04-27T08:20:02.440494622Z Llama.generate: prefix-match hit 2024-04-27T08:20:32.179235232Z TRACE: 127.0.0.1:37782 - HTTP connection lost 2024-04-27T08:21:53.660598108Z 2024-04-27T08:21:53.660731160Z llama_print_timings: load time = 8292.51 ms 2024-04-27T08:21:53.660738942Z llama_print_timings: sample time = 162.44 ms / 247 runs ( 0.66 ms per token, 1520.56 tokens per second) 2024-04-27T08:21:53.660744703Z llama_print_timings: prompt eval time = 0.00 ms / 1 tokens ( 0.00 ms per token, inf tokens per second) 2024-04-27T08:21:53.660749637Z llama_print_timings: eval time = 109906.85 ms / 247 runs ( 444.97 ms per token, 2.25 tokens per second) 2024-04-27T08:21:53.660754973Z llama_print_timings: total time = 111236.26 ms / 248 tokens 2024-04-27T08:21:53.661498139Z TRACE: 127.0.0.1:37782 - ASGI [3634] Send {'type': 'http.response.start', 'status': 200, 'headers': '<...>'} 2024-04-27T08:21:53.661548671Z TRACE: 127.0.0.1:37782 - ASGI [3634] Send {'type': 'http.response.body', 'body': '<1329 bytes>'} 2024-04-27T08:21:53.661650318Z TRACE: 127.0.0.1:37782 - ASGI [3634] Completed ```

Screenshots If applicable, add screenshots to help explain your problem.

Setup Details (please complete the following information):

Additional context I'm running the backend without gpu, so it takes some time to get the answer ready.

kyteinsky commented 5 months ago

Hello, maybe this error comes from the proxy (like nginx). Can you try removing the keep alive header proxy_set_header Connection "";

or set keepalive to false if you have a config like that? See if that helps.

    location / {
        proxy_pass http://stable29.local;
        set $upstream_keepalive false;
    }
socialize-IT commented 5 months ago

There is no proxy running. Only the apache that is serving nextcloud directly. Disabling keeAlive in the apache doesn't change anything..

kyteinsky commented 5 months ago

In that case, the culprit could be php's execution time limit, which is "coincidentally" 30 seconds by default.

Set this in your php.ini. The value is in seconds.

max_execution_time = 3000
socialize-IT commented 5 months ago

max_execution_time is 7200 seconds on my system.

socialize-IT commented 5 months ago

Alright.. Silly me. I just updated the dsp and now it works like a charm.

kyteinsky commented 5 months ago

Nice. Will add that to the whole thing to the readme.