Chronos model support

[mykeehu]

Describe the bug

I use the Chronos 13B as a chat, but it also often has control characters, such as \n\n### Scenario: \nuser: or \nassistant: I added these as stop characters, but for example the long_term_memory extension entries just appeared! So the chronos models are probably not yet ready for the webui.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Chat with Chronos 13B model, I use this: https://huggingface.co/elinas/chronos-13b-4bit

Screenshot

No response

Logs

Here is a part of character .json:

and waiting patiently for guidance.*\n\n[SusanNew's Memory]:\n\n0 days ago, Miki said:\n\"*While looking around to check nobody is watching,

System Info

NVIDIA RTX 3060 12 GB
i9-13900K
64 GB RAM
Windows 10 64-bit.

[ossirytk]

How exactly are you running this model? I have been running this in chat mode with alpaca template for a while now and it works just fine for me.

[mykeehu]

Interesting, because I used it in chat mode and chat-instruct mode. I had WizardLM template set up, but by default in chat mode there is no template and then these errors occur. I have the chat_generation_attempts set to 3 by default to get longer prompts, and then I get a bunch of inappropriate code. In chat mode the alpaca formatting is not automatically set.

Update 1: Here is an example. After the first round was done in office, it generated this nonsense for Jenny response that doesn't belong there:

Me: This phone conversation also ended well! You are awesome Jenny!
Jenny: *Smiling* Thank you, boss! I appreciate your kind words!

### Response:
Sure thing, here's some more content for you:
As you continue your journey through the forest, you eventually reach a clearing where a small house stands. The windows are lit up and smoke is pouring out of the chimney. You decide to investigate further and knock on the door.

An old woman answers the door, dressed in simple robes. She looks at you curiously but welcomes you inside. The interior of the house is warm and cozy, decorated with various artifacts and trinkets.

The old woman introduces herself as Gramma Greta and offers you some tea. As she prepares the drink, she tells you about the history of the area and the legends surrounding the forest.

Gramma Greta mentions that the forest has been known to contain strange creatures and phenomena. However, she assures you that as long as you follow certain rules, you should be safe during your stay.

She then warns you against entering the forbidden areas of the forest and reveals that anyone who disobeys these warnings may encounter dangerous consequences.

With that said, Gramma Greta hands you a mug of steaming hot tea and encourages you to sit down by the fireplace to rest and reflect on your adventure thus far.

If I set chat_generation_attempts to 1, it stops fine after the first text.

[ossirytk]

I think it does not always set the correct template. And some models respond differently to different templates because the are mixed models. You might want to check from the model card, what the suggested format or formats are. Or try switching between chat and char instruct. I have had a lot of success with Chronos and the alpaca format.

[mykeehu]

The joke is that the previous answer was produced after the alpaca format was set in chat mode, so even if the format is set, it has no effect on chat mode. For chat-instruct it may be successful, but unfortunately there is no format that can be set individually for chat mode :( I have also experienced that I get different results in chat and chat-instruct mode, so it would be nice if the model was supported in chat mode in webui.

[ossirytk]

I'm running Chronos almost exclusively on chat mode, so it really seems to me that you just missed a setting. Seems like the same version too. Unless there is some difference in me running the ggml bin directly.

[ossirytk]

And the template I use I a default template. I'm not using some edited special template. Just the vanilla alpaca template with chat mode.

[mykeehu]

These are my settings for the text generation and model. Here I have already tried to stop the unnecessary generation of the model with stopping strings. I don't have a unique alpaca template either, I use the factory one:

[ossirytk]

Cool, but what template did you set in the chat settings?

[mykeehu]

I can't use a separate chat template, I can only set the instruct which is on alpaca:

[ossirytk]

I'm not sure, what you men when you say you can't use chat template. I think this field resets if you switch to chat mode, but you can just then set it to alpaca. That's at least the vague recollection I have. Can't really check right now. These seem basically identical to my settings otherwise. Unless you are running some extensions.

[github-actions[bot]]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.