search

lmg-anon / mikupad

LLM Frontend in a single html file
https://lmg-anon.github.io/mikupad/mikupad.html
Creative Commons Zero v1.0 Universal
233 stars 26 forks source link

The prompt templates are missing their `eos_token`? #81

Closed jukofyork closed 3 months ago

jukofyork commented 3 months ago

The prompts should have </s> (or whatever their specific eos_token is) appended after the assistant's reply? eg:

USER: Write me the opening chapter of a 'Grimdark' stories trilogy in the style of Joe Abercrombie and Rob J Hayes. Use third-person omniscient POV. The boy king has just come to power, but the machinery of politics is moving fast... ASSISTANT: In the shadowed halls of the Blackspire ... his strings pulled by unseen hands in a dance that would end only in tragedy.</s>USER: continue ASSISTANT:...

I think Alpaca may be the only prompt template that doesn't use this (or at least it seems to screw the very few models that use multi-turn Alpaca like phind-codellama and some of the older wizard models).

Cool tool BTW - very useful for testing creative writing models with! :+1:

lmg-anon commented 3 months ago

That's right, they were indeed missing the eos_token, thanks for pointing that out. I think all prompt templates should be correct now.