airoboros-c34b-2.1 prompting

jondurbin / airoboros

Customizable implementation of the self-instruct paper.

Apache License 2.0

1.02k stars 71 forks source link

airoboros-c34b-2.1 prompting #29

Closed Chainfire closed 1 year ago

Chainfire commented 1 year ago

As noted on HuggingFace:

This model is a bit broken due to a prompt formatting bug in the training code! 2.2 will be available soon and should fix this

Can you describe what is wrong with the prompting? I'm having various issues but not sure if they are related. Most prominently, the model often just stops responding for a while in chats (repeated prompts).

Also, is there an ETA on 2.2 for this variant? Days? Weeks? Months? Up in the air?

jondurbin commented 1 year ago

Essentially, for any of the fine-tuning data with a custom system prompt, it was not adding it, nor was it prefixing the instructions with USER:. It still worked moderately well, surprisingly, but the training code has been fixed.

I have occasionally noticed blank responses as well, not entirely sure what the cause is yet.

The new versions are here:

normal: https://huggingface.co/jondurbin/airoboros-c34b-2.2
de-censored: https://huggingface.co/jondurbin/spicyboros-c34b-2.2

Be sure to update your prompt formatting to use newlines (if you are using the standard USER/ASSISTANT style):

A chat.
USER: {prompt}
ASSISTANT:

No newline or space after "ASSISTANT:"

Chainfire commented 1 year ago

Thanks. Not sure how I both searched and missed this apparently multiple times today. Will try again with the new files and verify my prompt.

Chainfire commented 1 year ago

Thanks, 2.2 seems to work significantly better. I've converted/uploaded it in EXL2 format too - https://huggingface.co/JorritJ/spicyboros-c34b-2.2-4.0bpw-h6-exl2 .

It still sometimes refuses to give anything but a blank answer. If anyone ever figures out why that is, I'd be happy to learn.