Free 7b models need to be told to think better in system prompt

[tarasglek]

The following addition to system prompt makes free models way smarter and actually useful.

• before answering rephrase the question using different and your own words to make sure you understand it

[tarasglek]

openai gpt gets annoyingly more verbose with that, but can counteract it with following combo..this still gets excellent results on 7b too

• before answering rephrase the question using different own words to clarify asks
• moar dry responses

[tarasglek]

@humphd what do you think about adding these 2 bullets to chatcraft default prompt?

[humphd]

I would remove them from the default prompt if they were there, so I'm not sure. This feels like personal, system prompt customization to me.

[tarasglek]

try using the free models with/without and get back to me :)

[humphd]

That's my point, I don't use the free models, so adding this to my default system prompt sucks.

[humphd]

Thinking about this more...let's update the code in https://github.com/tarasglek/chatcraft.org/blob/main/src/lib/system-prompt.ts so that if we're using the default provider, we add extra bits to the system prompt. This way we can better target the use case we care about.

[tarasglek]

I remember last time we had code like this I ended up with a lot of weird looking system prompts.

my other thought was to hack it serverside

[humphd]

That's perhaps cleaner. Can we have multiple system prompts in a chat? If so, you could inject a second one to do this before sending to openrouter.

[tarasglek]

Here is another prompt hack https://discord.com/channels/1078625661620592721/1121489946004430949/1222030693551439914 in this vein