"Exiting chain with error: invalid character '<' looking for beginning of value"

[cartergrobinson]

Issue:

Error "Exiting chain with error: invalid character '<' looking for beginning of value"

Description:

I have installed LLocalSearch on Ubuntu 22.04 using Docker and have successfully connected it to my Ollama instance running on the same server (not in Docker). The Ollama service is configured to listen on all interfaces and responds to API requests from other systems on the LAN.

Ollama Installation:

It can successfully connect to my Ollama instance on the same server (not in Docker). I can tell it works because it successfully used the API to pull all-minilm:v2. Also, I changed the Ollama service to listen on all interfaces (ex. Environment="OLLAMA_HOST=0.0.0.0:11434" in /etc/systemd/system/ollama.service. I verified that I get a response when I run this from another system on the LAN, which makes me think Ollama is fine:

curl http://192.168.0.80:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why is the sky blue?"
}'

Error

However, when I use the LLocalSearch interface at http://localhost:3000, any preset or longer chat prompt I choose results in the following error:

Exiting chain with error: invalid character '<' looking for beginning of value

The backend logs show:

2024/04/04 06:23:04 INFO Starting the server
Server started at http://localhost:8080
2024/04/04 06:25:16 INFO Creating new session session=418c6f73-c3a6-4378-82cb-2881e48dda83
2024/04/04 06:25:16 INFO Starting agent chain session=418c6f73-c3a6-4378-82cb-2881e48dda83 userQuery="{Prompt:how much does obsidian sync cost? MaxIterations:30 ModelName:knoopx/hermes-2-pro-mistral:7b-q8_0 Session:418c6f73-c3a6-4378-82cb-2881e48dda83}" startTime=2024-04-04T06:25:16.250Z
Error parsing the JSON: invalid character '<' looking for beginning of value
Exiting chain with error: invalid character '<' looking for beginning of value

Simple test queries that don't trigger a search seem to work fine. For example, if I write "test," I get the expected response:

Test
The question "test" is not specific enough for me to provide an accurate response or determine if a tool is necessary. Could you please rephrase your question or provide more context so I can better assist you?

Logs:

2024/04/04 06:53:57 INFO Starting agent chain session=e0584807-dc7f-465b-871f-5b7162cafea3 userQuery="{Prompt:test MaxIterations:30 ModelName:knoopx/hermes-2-pro-mistral:7b-q8_0 Session:e0584807-dc7f-465b-871f-5b7162cafea3}" startTime=2024-04-04T06:53:57.390Z
2024/04/04 06:53:58 mem messages [{
    1. Fromat your answer (after AI:) in markdown. 
    2. You have to use your tools to answer questions. 
    3. You have to provide the sources / links you've used to answer the quesion.
    4. You may use tools more than once.
    Question: test} { # Test
The question "test" is not specific enough for me to provide an accurate response or determine if a tool is necessary. Could you please rephrase the question or provide more context so that I can better assist you?<|im_end|> <nil>}]

Docker-compose

Here is my docker-compose configuration. All I've changed is the IP for Ollama.

services:
  backend:
    image: nilsherzig/llocalsearch-backend:latest
    environment:
      # the ip / url of YOUR Ollama server
      # CHANGE THIS
      - OLLAMA_HOST=http://192.168.0.80:11434

      # the url of the chroma db
      - CHROMA_DB_URL=http://chromadb:8000

      # the url of the redis db
      - SEARXNG_DOMAIN=http://searxng:8080

      # the maximum amount of iterations the agent will run to find your answer
      - MAX_ITERATIONS=30
    networks:
      - llm_network

  frontend:
    depends_on:
      - backend
    image: nilsherzig/llocalsearch-frontend:latest
    ports:
      - '3000:4173'
    networks:
      - llm_network

  chromadb:
    image: chromadb/chroma
    networks:
      - llm_network

  redis:
    image: docker.io/library/redis:alpine
    command: redis-server --save 30 1 --loglevel warning
    networks:
      - searxng
    volumes:
      - redis-data:/data
    cap_drop:
      - ALL
    cap_add:
      - SETGID
      - SETUID
      - DAC_OVERRIDE

  searxng:
    image: docker.io/searxng/searxng:latest
    networks:
      - searxng
      - llm_network
    volumes:
      - ./searxng:/etc/searxng:rw
    environment:
      - SEARXNG_BASE_URL=https://${SEARXNG_HOSTNAME:-localhost}/
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    logging:
      driver: 'json-file'
      options:
        max-size: '1m'
        max-file: '1'

networks:
  llm_network:
    driver: bridge
  searxng:
    ipam:
      driver: default

volumes:
  redis-data:

I have double-checked the configuration and connections between the components, and everything seems to be set up correctly.

Could you please help me identify the cause of this error and provide guidance on how to resolve it? Let me know if you need any additional information.

Thank you for your assistance!

[grigio]

I'm having the same issue, the error should came from here https://github.com/nilsherzig/LLocalSearch/blob/54bb8cdb6a96b5569d37a1b1b8e6fe46f42ddba1/backend/llm_tools/tool_websearch.go#L57

But I have also this in the logs, so it could be a searxng issue

frontend-1  |   ➜  Network: http://172.21.0.5:4173/
searxng-1   | 2024-04-04 08:10:12,197 ERROR:searx.engines.soundcloud: Fail to initialize
searxng-1   | Traceback (most recent call last):
searxng-1   |   File "/usr/local/searxng/searx/network/__init__.py", line 95, in request
searxng-1   |     return future.result(timeout)
searxng-1   |            ^^^^^^^^^^^^^^^^^^^^^^
searxng-1   |   File "/usr/lib/python3.11/concurrent/futures/_base.py", line 458, in result
searxng-1   |     raise TimeoutError()
searxng-1   | TimeoutError
searxng-1   | 
searxng-1   | The above exception was the direct cause of the following exception:
searxng-1   | 
searxng-1   | Traceback (most recent call last):
searxng-1   |   File "/usr/local/searxng/searx/search/processors/abstract.py", line 73, in initialize
searxng-1   |     self.engine.init(get_engine_from_settings(self.engine_name))
searxng-1   |   File "/usr/local/searxng/searx/engines/soundcloud.py", line 69, in init
searxng-1   |     guest_client_id = get_client_id()
searxng-1   |                       ^^^^^^^^^^^^^^^
searxng-1   |   File "/usr/local/searxng/searx/engines/soundcloud.py", line 57, in get_client_id

[nilsherzig]

Hi could you show me your searxng settings? You need to enable json responses.

https://github.com/nilsherzig/LLocalSearch/blob/50646ebfccb05e0b3daa12ba17fc622f3810b1d7/searxng/settings.yml#L25-L27

[cartergrobinson]

This was in the default file (/data/compose/2/searxng/settings.yml) from the container in the compose file.

  # remove format to deny access, use lower case.
  # formats: [html, csv, json, rss]
  formats:
    - html

Manually changed to:

  # remove format to deny access, use lower case.
  # formats: [html, csv, json, rss]
  formats:
    - html
    - json

Restarted containers, and now it's good.

[nilsherzig]

Ah yes, you've changed the searxng volume. This setting is the reason why i put my searxng folder in this github repo haha. I think it should be possible to change the searxng start comamnd to something, which will patch the default file and start searxng after that. This would allow us to remove the folder / requirement for a volume