Not able to run code based on examples/lightrag_ollama_demo.py

[32bitmicro]

When I run code based on examples/lightrag_ollama_demo.py I get following error, see below. This happens withllm_model_name="gemma2:2b"

python3.11/site-packages/tiktoken/model.py", line 84, in encoding_name_for_model
    raise KeyError(
KeyError: 'Could not automatically map gpt-4o-mini to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.'

Can you suggest a solution?

[Jaykumaran]

Hello,

I didnt face error with lightrag_ollama_demo.py, when i used the sample code from repo using gemma2:2b recently.

I created a new env, followed setup procedure as listed under readme.

Then ollama pull gemma2:2b . Adjust your books.txt relative path.

import os
import logging
from lightrag import LightRAG, QueryParam
from lightrag.llm import ollama_model_complete, ollama_embedding
from lightrag.utils import EmbeddingFunc

WORKING_DIR = "./dickens"

logging.basicConfig(format="%(levelname)s:%(message)s", level=logging.INFO)

if not os.path.exists(WORKING_DIR):
    os.mkdir(WORKING_DIR)

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=ollama_model_complete,
    llm_model_name="gemma2:2b",
    llm_model_max_async=4,
    llm_model_max_token_size=32000,
    llm_model_kwargs={"host": "http://localhost:11434", "options": {"num_ctx": 32768}},
    embedding_func=EmbeddingFunc(
        embedding_dim=768,
        max_token_size=8192,
        func=lambda texts: ollama_embedding(
            texts, embed_model="nomic-embed-text", host="http://localhost:11434"
        ),
    ),
)

with open("../book.txt", "r", encoding="utf-8") as f:
    rag.insert(f.read())

# Perform naive search
print(
    rag.query("What are the top themes in this story?", param=QueryParam(mode="naive"))
)

# Perform local search
print(
    rag.query("What are the top themes in this story?", param=QueryParam(mode="local"))
)

# Perform global search
print(
    rag.query("What are the top themes in this story?", param=QueryParam(mode="global"))
)

# Perform hybrid search
print(
    rag.query("What are the top themes in this story?", param=QueryParam(mode="hybrid"))
)

Can you be more specific about your issue and share your script.

P.S. I faced an obvious issue of being extremely slow when working with ollama models with my lap RTX4050. However i managed to run with my pc.

[32bitmicro]

Well the error is about wrong tokeniser in my python 3.11 environment maybe that is the problem. What python version do you use?
My script references local docs, I will try with what you have posted and let you know. I actually got it running by adding this line: tiktoken_model_name="gpt2" just before embedding_func=EmbeddingFunc( But it did not seem to make any progress and I had to kill it after couple of hours. How long was it running on your PC?

[32bitmicro]

Here is the result, still seeing the error INFO:lightrag:Writing graph with 0 nodes, 0 edges Traceback (most recent call last): File ./rag_ollama_demo.py", line 31, in rag.insert(f.read()) File "miniconda3/lib/python3.11/site-packages/lightrag/lightrag.py", line 167, in insert return loop.run_until_complete(self.ainsert(string_or_strings)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "miniconda3/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "miniconda3/lib/python3.11/site-packages/lightrag/lightrag.py", line 192, in ainsert for dp in chunking_by_token_size( ^^^^^^^^^^^^^^^^^^^^^^^ File "miniconda3/lib/python3.11/site-packages/lightrag/operate.py", line 32, in chunking_by_token_size tokens = encode_string_by_tiktoken(content, model_name=tiktoken_model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "miniconda3/lib/python3.11/site-packages/lightrag/utils.py", line 121, in encode_string_by_tiktoken ENCODER = tiktoken.encoding_for_model(model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "miniconda3/lib/python3.11/site-packages/tiktoken/model.py", line 97, in encoding_for_model return get_encoding(encoding_name_for_model(model_name)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "miniconda3/lib/python3.11/site-packages/tiktoken/model.py", line 84, in encoding_name_for_model raise KeyError( KeyError: 'Could not automatically map gpt-4o-mini to a tokeniser. Please use tiktoken.get_encoding to explicitly get the tokeniser you expect.'