search

ItzCrazyKns / Perplexica

Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI
MIT License
14.4k stars 1.39k forks source link

Parts of the UI not working, "WebSocket is closed before the connection is established" #269

Closed manos-msdn closed 3 months ago

manos-msdn commented 3 months ago

Hi, I just checked out the repo and started the docker containers, most of the stuff is working, but some part of the UI is not, any ideas where to look next? My gut feeling is it's something about my networking settings or sockets/certificate issue?

Running on windows 10 with docker desktop on wsl 2 and "docker.internal" hostnames enabled, against a local ollama3 model

The model gets discovered correctly by the front end, so the ollama and the api part seem to be working, also the settings part of the UI is working fine, but the rest of the pages (discover, chat, library) are not. The search is working (though it has some errors about gentoo module not found in the docker logs, but I don't think that's the issues as I can perform searches fine).

When going to http://localhost:3000 this is the error:

image

Containers running:

image

Ollama working:

image

Search working

image

API working

image

Settings page working

image

My settings files:

toml 

[GENERAL]
PORT = 3001 # Port to run the server on
SIMILARITY_MEASURE = "cosine" # "cosine" or "dot"

[API_KEYS]
OPENAI = "" # OpenAI API key - sk-1234567890abcdef1234567890abcdef
GROQ = "" # Groq API key - gsk_1234567890abcdef1234567890abcdef

[API_ENDPOINTS]
SEARXNG = "http://host.docker.internal:4000"  # "http://localhost:32768" # SearxNG API URL
OLLAMA = "http://host.docker.internal:11434" # Ollama API URL - http://host.docker.internal:11434

compose:

services:
  searxng:
    image: docker.io/searxng/searxng:latest
    volumes:
      - ./searxng:/etc/searxng:rw
    ports:
      - 4000:8080
    networks:
      - perplexica-network
    restart: unless-stopped

  perplexica-backend:
    build:
      context: .
      dockerfile: backend.dockerfile
      args:
        - SEARXNG_API_URL=http://host.docker.internal:8080
    depends_on:
      - searxng
    ports:
      - 3001:3001
    volumes:
      - backend-dbstore:/home/perplexica/data
    extra_hosts:
      - 'host.docker.internal:host-gateway'
    networks:
      - perplexica-network
    restart: unless-stopped

  perplexica-frontend:
    build:
      context: .
      dockerfile: app.dockerfile
      args:
        - NEXT_PUBLIC_API_URL=http://host.docker.internal:3001/api
        - NEXT_PUBLIC_WS_URL=ws://host.docker.internal:3001
    depends_on:
      - perplexica-backend
    ports:
      - 3000:3000
    networks:
      - perplexica-network
    restart: unless-stopped

networks:
  perplexica-network:

volumes:
  backend-dbstore:

docker logs:

docker compose logs perplexica-frontend-1 | yarn run v1.22.22 searxng-1 | SearXNG version 2024.7.14+e56f4b315 perplexica-frontend-1 | $ next start searxng-1 | Use existing /etc/searxng/uwsgi.ini perplexica-frontend-1 | ▲ Next.js 14.1.4 searxng-1 | Use existing /etc/searxng/settings.yml perplexica-frontend-1 | - Local: http://localhost:3000 perplexica-frontend-1 | searxng-1 | Listen on 0.0.0.0:8080 perplexica-frontend-1 | ✓ Ready in 1931ms searxng-1 | [uWSGI] getting INI configuration from /etc/searxng/uwsgi.ini searxng-1 | [uwsgi-static] added mapping for /static => /usr/local/searxng/searx/static searxng-1 | Starting uWSGI 2.0.25.1 (64bit) on [Sun Jul 14 17:19:15 2024] searxng-1 | compiled with version: 13.2.1 20240309 on 17 May 2024 06:11:37 searxng-1 | os: Linux-5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 searxng-1 | nodename: 36415a0b241f searxng-1 | machine: x86_64 searxng-1 | clock source: unix searxng-1 | pcre jit disabled searxng-1 | detected number of CPU cores: 2 searxng-1 | current working directory: /usr/local/searxng searxng-1 | detected binary path: /usr/sbin/uwsgi searxng-1 | chdir() to /usr/local/searxng/searx/ searxng-1 | your memory page size is 4096 bytes searxng-1 | detected max file descriptor number: 1048576 searxng-1 | building mime-types dictionary from file /etc/mime.types...1390 entry found searxng-1 | lock engine: pthread robust mutexes searxng-1 | thunder lock: disabled (you can enable it with --thunder-lock) searxng-1 | uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 3 searxng-1 | Python version: 3.12.3 (main, Apr 18 2024, 07:52:31) [GCC 13.2.1 20240309] searxng-1 | Python main interpreter initialized at 0x7ffa75079688 searxng-1 | python threads support enabled searxng-1 | your server socket listen backlog is limited to 100 connections searxng-1 | your mercy for graceful operations on workers is 60 seconds searxng-1 | mapped 362016 bytes (353 KB) for 8 cores searxng-1 | Operational MODE: preforking+threaded perplexica-backend-1 | yarn run v1.22.22 perplexica-backend-1 | $ npm run db:push && node dist/app.js perplexica-backend-1 | perplexica-backend-1 | > perplexica-backend@1.7.1 db:push perplexica-backend-1 | > drizzle-kit push sqlite perplexica-backend-1 | perplexica-backend-1 | drizzle-kit: v0.22.7 perplexica-backend-1 | drizzle-orm: v0.31.2 searxng-1 | added /usr/local/searxng/ to pythonpath. searxng-1 | spawned uWSGI master process (pid: 7) searxng-1 | spawned uWSGI worker 1 (pid: 10, cores: 4) searxng-1 | spawned uWSGI worker 2 (pid: 11, cores: 4) searxng-1 | spawned 4 offload threads for uWSGI worker 2 searxng-1 | spawned 4 offload threads for uWSGI worker 1 searxng-1 | 2024-07-14 17:19:19,565 ERROR:searx.engines: Cannot load engine "gentoo" searxng-1 | Traceback (most recent call last): searxng-1 | File "/usr/local/searxng/searx/engines/init.py", line 122, in load_engine searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/utils.py", line 437, in load_module searxng-1 | spec.loader.exec_module(module) searxng-1 | File "", line 991, in exec_module perplexica-backend-1 | searxng-1 | File "", line 1128, in get_code perplexica-backend-1 | No config path provided, using default path searxng-1 | File "", line 1186, in get_data perplexica-backend-1 | Reading config file '/home/perplexica/drizzle.config.ts' searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/gentoo.py' perplexica-backend-1 | [⣷] Pulling schema from database... [✓] Pulling schema from database... perplexica-backend-1 | searxng-1 | 2024-07-14 17:19:19,575 ERROR:searx.engines: Cannot load engine "gentoo" perplexica-backend-1 | [i] No changes detected perplexica-backend-1 | npm notice perplexica-backend-1 | npm notice New patch version of npm available! 10.8.1 -> 10.8.2 searxng-1 | Traceback (most recent call last): perplexica-backend-1 | npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2 perplexica-backend-1 | npm notice To update run: npm install -g npm@10.8.2 perplexica-backend-1 | npm notice perplexica-backend-1 | info: WebSocket server started on port 3001 searxng-1 | File "/usr/local/searxng/searx/engines/init.py", line 122, in load_engine searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/utils.py", line 437, in load_module perplexica-backend-1 | (node:59) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead. searxng-1 | spec.loader.exec_module(module) searxng-1 | File "", line 991, in exec_module perplexica-backend-1 | (Use node --trace-deprecation ... to show where the warning was created) searxng-1 | File "", line 1128, in get_code perplexica-backend-1 | info: Server is running on port 3001 searxng-1 | File "", line 1186, in get_data searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/gentoo.py' searxng-1 | WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x7ffa75079688 pid: 10 (default app) searxng-1 | WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x7ffa75079688 pid: 11 (default app) searxng-1 | 2024-07-14 17:31:24,929 WARNING:searx.engines.tootfinder: ErrorContext('searx/search/processors/online.py', 116, "response = req(params['url'], request_args)", 'searx.exceptions.SearxEngineAccessDeniedException', None, ('HTTP error 403',)) False searxng-1 | 2024-07-14 17:31:24,938 ERROR:searx.engines.tootfinder: SearXNG is blocked searxng-1 | Traceback (most recent call last): searxng-1 | File "/usr/local/searxng/searx/search/processors/online.py", line 163, in search searxng-1 | search_results = self._search_basic(query, params) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/search/processors/online.py", line 147, in _search_basic
searxng-1 | response = self._send_http_request(params) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/search/processors/online.py", line 116, in _send_http_request searxng-1 | response = req(params['url'], request_args) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/init.py", line 164, in get searxng-1 | return request('get', url, kwargs) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/init.py", line 95, in request searxng-1 | return future.result(timeout) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result searxng-1 | return self.get_result() searxng-1 | ^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in get_result searxng-1 | raise self._exception searxng-1 | File "/usr/local/searxng/searx/network/network.py", line 289, in request searxng-1 | return await self.call_client(False, method, url, kwargs) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/network.py", line 272, in call_client searxng-1 | return Network.patch_response(response, do_raise_for_httperror) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/network.py", line 245, in patch_response searxng-1 | raise_for_httperror(response) searxng-1 | File "/usr/local/searxng/searx/network/raise_for_httperror.py", line 74, in raise_for_httperror searxng-1 | raise SearxEngineAccessDeniedException(message='HTTP error ' + str(resp.status_code))
searxng-1 | searx.exceptions.SearxEngineAccessDeniedException: HTTP error 403, suspended_time=86400

ItzCrazyKns commented 3 months ago

Hi, I just checked out the repo and started the docker containers, most of the stuff is working, but some part of the UI is not, any ideas where to look next? My gut feeling is it's something about my networking settings or sockets/certificate issue?

Running on windows 10 with docker desktop on wsl 2 and "docker.internal" hostnames enabled, against a local ollama3 model

The model gets discovered correctly by the front end, so the ollama and the api part seem to be working, also the settings part of the UI is working fine, but the rest of the pages (discover, chat, library) are not. The search is working (though it has some errors about gentoo module not found in the docker logs, but I don't think that's the issues as I can perform searches fine).

When going to http://localhost:3000 this is the error:

image

Containers running:

image

Ollama working:

image

Search working

image

API working

image

Settings page working

image

My settings files:

toml

[GENERAL]
PORT = 3001 # Port to run the server on
SIMILARITY_MEASURE = "cosine" # "cosine" or "dot"

[API_KEYS]
OPENAI = "" # OpenAI API key - sk-1234567890abcdef1234567890abcdef
GROQ = "" # Groq API key - gsk_1234567890abcdef1234567890abcdef

[API_ENDPOINTS]
SEARXNG = "http://host.docker.internal:4000"  # "http://localhost:32768" # SearxNG API URL
OLLAMA = "http://host.docker.internal:11434" # Ollama API URL - http://host.docker.internal:11434

compose:

services:
  searxng:
    image: docker.io/searxng/searxng:latest
    volumes:
      - ./searxng:/etc/searxng:rw
    ports:
      - 4000:8080
    networks:
      - perplexica-network
    restart: unless-stopped

  perplexica-backend:
    build:
      context: .
      dockerfile: backend.dockerfile
      args:
        - SEARXNG_API_URL=http://host.docker.internal:8080
    depends_on:
      - searxng
    ports:
      - 3001:3001
    volumes:
      - backend-dbstore:/home/perplexica/data
    extra_hosts:
      - 'host.docker.internal:host-gateway'
    networks:
      - perplexica-network
    restart: unless-stopped

  perplexica-frontend:
    build:
      context: .
      dockerfile: app.dockerfile
      args:
        - NEXT_PUBLIC_API_URL=http://host.docker.internal:3001/api
        - NEXT_PUBLIC_WS_URL=ws://host.docker.internal:3001
    depends_on:
      - perplexica-backend
    ports:
      - 3000:3000
    networks:
      - perplexica-network
    restart: unless-stopped

networks:
  perplexica-network:

volumes:
  backend-dbstore:

docker logs:

docker compose logs perplexica-frontend-1 | yarn run v1.22.22 searxng-1 | SearXNG version 2024.7.14+e56f4b315 perplexica-frontend-1 | $ next start searxng-1 | Use existing /etc/searxng/uwsgi.ini perplexica-frontend-1 | ▲ Next.js 14.1.4 searxng-1 | Use existing /etc/searxng/settings.yml perplexica-frontend-1 | - Local: http://localhost:3000 perplexica-frontend-1 | searxng-1 | Listen on 0.0.0.0:8080 perplexica-frontend-1 | ✓ Ready in 1931ms searxng-1 | [uWSGI] getting INI configuration from /etc/searxng/uwsgi.ini searxng-1 | [uwsgi-static] added mapping for /static => /usr/local/searxng/searx/static searxng-1 | Starting uWSGI 2.0.25.1 (64bit) on [Sun Jul 14 17:19:15 2024] searxng-1 | compiled with version: 13.2.1 20240309 on 17 May 2024 06:11:37 searxng-1 | os: Linux-5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 searxng-1 | nodename: 36415a0b241f searxng-1 | machine: x86_64 searxng-1 | clock source: unix searxng-1 | pcre jit disabled searxng-1 | detected number of CPU cores: 2 searxng-1 | current working directory: /usr/local/searxng searxng-1 | detected binary path: /usr/sbin/uwsgi searxng-1 | chdir() to /usr/local/searxng/searx/ searxng-1 | your memory page size is 4096 bytes searxng-1 | detected max file descriptor number: 1048576 searxng-1 | building mime-types dictionary from file /etc/mime.types...1390 entry found searxng-1 | lock engine: pthread robust mutexes searxng-1 | thunder lock: disabled (you can enable it with --thunder-lock) searxng-1 | uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 3 searxng-1 | Python version: 3.12.3 (main, Apr 18 2024, 07:52:31) [GCC 13.2.1 20240309] searxng-1 | Python main interpreter initialized at 0x7ffa75079688 searxng-1 | python threads support enabled searxng-1 | your server socket listen backlog is limited to 100 connections searxng-1 | your mercy for graceful operations on workers is 60 seconds searxng-1 | mapped 362016 bytes (353 KB) for 8 cores searxng-1 | Operational MODE: preforking+threaded perplexica-backend-1 | yarn run v1.22.22 perplexica-backend-1 | $ npm run db:push && node dist/app.js perplexica-backend-1 | perplexica-backend-1 | > perplexica-backend@1.7.1 db:push perplexica-backend-1 | > drizzle-kit push sqlite perplexica-backend-1 | perplexica-backend-1 | drizzle-kit: v0.22.7 perplexica-backend-1 | drizzle-orm: v0.31.2 searxng-1 | added /usr/local/searxng/ to pythonpath. searxng-1 | spawned uWSGI master process (pid: 7) searxng-1 | spawned uWSGI worker 1 (pid: 10, cores: 4) searxng-1 | spawned uWSGI worker 2 (pid: 11, cores: 4) searxng-1 | spawned 4 offload threads for uWSGI worker 2 searxng-1 | spawned 4 offload threads for uWSGI worker 1 searxng-1 | 2024-07-14 17:19:19,565 ERROR:searx.engines: Cannot load engine "gentoo" searxng-1 | Traceback (most recent call last): searxng-1 | File "/usr/local/searxng/searx/engines/init.py", line 122, in load_engine searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/utils.py", line 437, in load_module searxng-1 | spec.loader.exec_module(module) searxng-1 | File "", line 991, in exec_module perplexica-backend-1 | searxng-1 | File "", line 1128, in get_code perplexica-backend-1 | No config path provided, using default path searxng-1 | File "", line 1186, in get_data perplexica-backend-1 | Reading config file '/home/perplexica/drizzle.config.ts' searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/gentoo.py' perplexica-backend-1 | [⣷] Pulling schema from database... [✓] Pulling schema from database... perplexica-backend-1 | searxng-1 | 2024-07-14 17:19:19,575 ERROR:searx.engines: Cannot load engine "gentoo" perplexica-backend-1 | [i] No changes detected perplexica-backend-1 | npm notice perplexica-backend-1 | npm notice New patch version of npm available! 10.8.1 -> 10.8.2 searxng-1 | Traceback (most recent call last): perplexica-backend-1 | npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.2 perplexica-backend-1 | npm notice To update run: npm install -g npm@10.8.2 perplexica-backend-1 | npm notice perplexica-backend-1 | info: WebSocket server started on port 3001 searxng-1 | File "/usr/local/searxng/searx/engines/init.py", line 122, in load_engine searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/utils.py", line 437, in load_module perplexica-backend-1 | (node:59) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead. searxng-1 | spec.loader.exec_module(module) searxng-1 | File "", line 991, in exec_module perplexica-backend-1 | (Use node --trace-deprecation ... to show where the warning was created) searxng-1 | File "", line 1128, in get_code perplexica-backend-1 | info: Server is running on port 3001 searxng-1 | File "", line 1186, in get_data searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/gentoo.py' searxng-1 | WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x7ffa75079688 pid: 10 (default app) searxng-1 | WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x7ffa75079688 pid: 11 (default app) searxng-1 | 2024-07-14 17:31:24,929 WARNING:searx.engines.tootfinder: ErrorContext('searx/search/processors/online.py', 116, "response = req(params['url'], request_args)", 'searx.exceptions.SearxEngineAccessDeniedException', None, ('HTTP error 403',)) False searxng-1 | 2024-07-14 17:31:24,938 ERROR:searx.engines.tootfinder: SearXNG is blocked searxng-1 | Traceback (most recent call last): searxng-1 | File "/usr/local/searxng/searx/search/processors/online.py", line 163, in search searxng-1 | search_results = self._search_basic(query, params) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/search/processors/online.py", line 147, in _search_basic searxng-1 | response = self._send_http_request(params) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/search/processors/online.py", line 116, in _send_http_request searxng-1 | response = req(params['url'], request_args) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/init.py", line 164, in get searxng-1 | return request('get', url, kwargs) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/init.py", line 95, in request searxng-1 | return future.result(timeout) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result searxng-1 | return self.get_result() searxng-1 | ^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in get_result searxng-1 | raise self._exception searxng-1 | File "/usr/local/searxng/searx/network/network.py", line 289, in request searxng-1 | return await self.call_client(False, method, url, kwargs) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/network.py", line 272, in call_client searxng-1 | return Network.patch_response(response, do_raise_for_httperror) searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ searxng-1 | File "/usr/local/searxng/searx/network/network.py", line 245, in patch_response searxng-1 | raise_for_httperror(response) searxng-1 | File "/usr/local/searxng/searx/network/raise_for_httperror.py", line 74, in raise_for_httperror searxng-1 | raise SearxEngineAccessDeniedException(message='HTTP error ' + str(resp.status_code)) searxng-1 | searx.exceptions.SearxEngineAccessDeniedException: HTTP error 403, suspended_time=86400

I pushed a fix for this (yesterday I believe), I'd recommend you to re-clone the project and follow the installation steps again. Make sure to delete the previous images before building new one or you can directly run this command in place of the last command

docker compose up -d --build
manos-msdn commented 3 months ago

thanks @ItzCrazyKns - I confirm this worked (commit f02393dbe9c18cd5e19df43f3698de30fbb418e1),

the chat and library work properly, just the discover link still gives 404 for some weird reason, not a big issue, just thought to let you know

image

ItzCrazyKns commented 3 months ago

The discover feature is under development; the page isn't implemented yet.