search

bdambrosio / AllTheWorldAPlay

All the world is a play, we are but actors in it.
46 stars 5 forks source link

Doesn't work with tabby #1

Open bdambrosio opened 5 months ago

bdambrosio commented 5 months ago

Well, it works, but text out is poor quality.

bdambrosio commented 5 months ago

I think the issue is different handling of tokeniser_config chat templates by tabby vs hf's tokenizer. In particular, llama-3 template doesn't initialize 'add_generation_prompt'. tabby seems to default to True. But my prompts, at least, run much better in llama-3 variants without that final empty Assistant message.