Getting the following error GGML_ASSERT(!grammar->stacks.empty()) failed

nidhinkumar06 commented 2 weeks ago

System Info / 系統信息

I have installed the Longwriter and tried running it where i am getting the following error message

GGML_ASSERT(!grammar->stacks.empty()) failed

As well as sometimes i get the response as !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Here is my implmentation using ollama

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

[ ] The official example scripts / 官方的示例脚本
[X] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

Pls find the implmentation code which i have used


`MODEL = "EntropyYue/longwriter-glm4:9b"
client = Client(host="http://xyz.*********")

messages = [
    {"role": "user", "content": "User Question : " + query + "\n\n CONTEXTS :\n\n" + contexts}
]

stream = client.chat(
            model=MODEL,
            messages=messages,
            stream=True,
            options={
              "temperature": 0,
            }
        )

        for chunk in stream:
            # print("chunk is", chunk)
            if chunk["message"]["content"] is not None:
                yield chunk["message"]["content"]

    except Exception as e:
        print(f"Error during get_answer_groq call: {e}")
        yield "data:" + json.dumps(
            {'type': 'error', 'data': "We are currently experiencing some issues. Please try again later."}) + "\n\n"``

### Expected behavior / 期待表现

It should provide the result unfortunately getting the error like GGML_ASSERT(!grammar->stacks.empty()) failed as well as sometimes the response as !!!!!!!!!!!!!!!!!!!!!!!!!!!!

bys0318 commented 1 week ago

Hi, I suspect a misalignment in the chat prompt template, but I'm not sure how client.chat deal with the chat template. Can you provide more details?

nidhinkumar06 commented 6 days ago

Hi one of the example context which is passed is like below

[{'role': 'user', 'content': 'User Question : hello\n\n CONTEXTS :\n\nHELLO! brings you the latest celebrity & royal news from the UK & around the world, magazine exclusives, fashion, beauty, lifestyle news, celeb babies, ... \n\n"Hello" is a song recorded by English singer-songwriter Adele, released on 23 October 2015 by XL Recordings as the lead single from her third studio album, ... \n\nREMASTERED IN HD! Explore the music of Lionel Richie: https://lnk.to/LionelBestOf Watch more ... 20 Nov 2020\n\nListen to "Easy On Me" here: http://Adele.lnk.to/EOM Pre-order Adele\'s new album "30" before ... 9 Nov 2015\n\nHELLO definition: 1. used when meeting or greeting someone: 2. something that is said at the beginning of a phone…. Learn more. \n\nThe meaning of HELLO is an expression or gesture of greeting —used interjectionally in greeting, in answering the telephone, or to express ... 13 Sept 2024\n\nWe are pausing hello while we get ready to launch something even better. We have loved having you a part of this community, and we want to thank you for all you ... \n\nStart off your lesson with “Hello!”, a fun and energetic song to talk about how you feel as you greet each other. \n\nBoost your website\'s conversions with Hello Bar\'s suite of marketing tools. Discover how to enhance visitor engagement today! \n\nAqyila #Hello Aqyila – Hello (Official Audio) Listen Now: (https://Aqyila.lnk.to/HelloAY) Follow ... 30 Mar 2023\n\nYou can use the Settings app to configure and manage Windows Hello. In the Settings app on your Windows device, select Accounts > Sign-in options. \n\nHello Theme: Fast, Lightweight, and FREE. Hello theme is built with minimal styling and scripts for maximum speed and design freedom. \n\nMeet iX Hello, our GenAI-powered virtual assistant that boosts productivity with customizable, no-code deployment. \n\nHello ... For other uses, see Hello (disambiguation). Hello is a salutation or greeting in the English language. It is first attested in writing from 1826. \n\nHello, hello, hello, my name is Ali. Hello, hello, hello, I\'m nine years old. Hello, hello, hello, I\'m pleased to meet you. This is my sister. This is my ... \n\nHello definition: (used to express a greeting, answer a telephone, or attract attention.). See examples of HELLO used in a sentence. \n\nHello Elementor is a lightweight and minimalist WordPress theme that was built specifically to work seamlessly with the Elementor site ... 30 Jul 2024\n\nShare your videos with friends, family, and the world. '}]

THUDM / LongWriter