search

ollama / ollama

Get up and running with Llama 3, Mistral, Gemma 2, and other large language models.
https://ollama.com
MIT License
78.83k stars 6k forks source link

InternLM2.5 - hallucinations - lot of repetitions etc #5474

Open Qualzz opened 1 week ago

Qualzz commented 1 week ago

What is the issue?

Seems like something is wrong with InternLM2.5, I can't get any meaningful out of it. (tried with 32k context)

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

v0.1.48

ELigoP commented 1 week ago

Could this be related? I use QuantFactory/internlm2_5-7b-chat-1m-GGUF quantized model with LMStudio. image

Qualzz commented 1 week ago

Something seems broken with either the GGUF or the implementation yup.

jmorganca commented 1 week ago

Hi @Qualzz which prompt are you using? I'm seeing okay results with that context window:

 ollama run internlm2
>>> /set parameter num_ctx 32768
Set parameter 'num_ctx' to '32768'
>>> Write me a long story
**Title: The Echoes of Eternity**

In the heart of a realm where time flowed like water through veins, there existed an enigmatic place known as Eldoria. A land bathed in perpetual
twilight and whispered secrets by the silvery whispers of the moonbeams. Here, amongst towering crystal spires that pierced the heavens, dwell the
guardians—ancient beings who preserve the balance between life's myriad paths.

Let me know - can look into this

Qualzz commented 1 week ago

I don't have an exemple right now, but it's similar to the screenshot posted Eligop Especially for long prompt and asking to summary, that kind of stuff.