Default Template Causes Longer Lists to Become Incoherent

[mutewinter]

I used the default front matter template provided in the repo, modified for GPT-4:

temperature: 0.9
top_p: 1
max_tokens: 512
presence_penalty: 1
frequency_penalty: 1
stream: true
stop: null
n: 1
model: gpt-4

I ran into issues with output of list-style responses like so:

Near the end of the list, the formatting and text starts to become incoherent.

---

After a bit of testing, I discovered that frequency_penalty: 1 was not the default for the API, which is 0. Upon setting frequency_penalty to 0, lists became cleaner (and more closely match the output from ChatGPT).

I was able to reproduce this same issue with the GPT 3.5 Turbo model.

Possible Fixes

• Include a default with lower or zero penalties.
• Include many template options with pros and cons listed.

[mutewinter]

I have since updated my GPT-4 template to be more concise and as close to the OpenAPI defaults as possible:

model: gpt-4
max_tokens: 1024
temperature: 1
presence_penalty: 0
frequency_penalty: 0
top_p: 1
n: 1

[tophee]

@bramses Any particular reason why you chose to change the default penalty?

[bramses]

@tophee not particularly haha. I can change both in default frontmatter to 0.

[bramses]

@mutewinter temp generally has an effect on long completions as well, but good point, I'll match the OpenAI recommendations then people can deviate from there.

In general though, the purpose of the default frontmatter is for users to choose what works for them