Ollama as provider

[sanzhar-rakhimkul]

Hello. When I try to use Ollama as LLM provider, I have following error. I used your example for deepseek.

async def deepseepk_model_if_cache(
    prompt, system_prompt=None, history_messages=[], **kwargs
) -> str:
    openai_async_client = AsyncOpenAI(
        base_url="http://10.204.54.11:11434/v1/",
        api_key="ollama",
    )
    messages = []
    if system_prompt:
        messages.append({"role": "system", "content": system_prompt})

    # Get the cached response if having-------------------
    hashing_kv: BaseKVStorage = kwargs.pop("hashing_kv", None)
    messages.extend(history_messages)
    messages.append({"role": "user", "content": prompt})
    if hashing_kv is not None:
        args_hash = compute_args_hash(MODEL, messages)
        if_cache_return = await hashing_kv.get_by_id(args_hash)
        if if_cache_return is not None:
            return if_cache_return["return"]
    # -----------------------------------------------------

    response = await openai_async_client.chat.completions.create(
        model=MODEL, messages=messages, **kwargs
    )

    # Cache the response if having-------------------
    if hashing_kv is not None:
        await hashing_kv.upsert({
            args_hash: {"return": response.choices[0].message.content, "model": MODEL}
        })
    # -----------------------------------------------------
    return response.choices[0].message.content

For some reason my embeddings_list = [] is empty and I get the error. Could you please help me │ /home/robot/Code/repo/nano-graphrag/nano_graphrag/_storage.py:94 in upsert │ │ │ │ 91 │ │ embeddings_list = await asyncio.gather( │ │ 92 │ │ │ *[self.embedding_func(batch) for batch in batches] │ │ 93 │ │ ) │ │ ❱ 94 │ │ embeddings = np.concatenate(embeddings_list) │ │ 95 │ │ for i, d in enumerate(list_data): │ │ 96 │ │ │ d["vector"] = embeddings[i] │ │ 97 │ │ results = self._client.upsert(datas=list_data) │ │

ValueError: need at least one array to concatenate

Full logs are here: output.log

[gusye1234]

Hi! Are you sure you pass the right text to insert? It seems not a LLM issue

[gusye1234]

Or what LLM you are using? Maybe the LLM is unable to follow the instruction and extract any entities, so the list is empty and trigger this error.

[sanzhar-rakhimkul]

I just checked the text, it is not empty. I previously tried the text with gpt-4o it worked perfectly.

I just tried:

• sciphi/triplex:latest q4, q8
• mistral-larg:123b-instruct-2407-q8_0
• llama3.1:70b-q8. Nothing worked.

[gusye1234]

Emmm, that's weird. llama3.1 70B shouldn't be that bad. I will add a new commit that will warning you when there is no new entities.

[sanzhar-rakhimkul]

I don't know what was the problem. I just deleted WORKING_DIR and started again. Now it works. Thanks for a help