Change the default config clipboard START_SEQ and END_SEQ to "[CLIPSTART]" and "[CLIPEND]" for better reliability.
To find this, I ran multiple styles through a benchmark of 20 different questions repeated 200 times for each style.
Model: Llama 3 8B
START_SEQ
END_SEQ
Success rate (%)
-CLIPSTART-
-CLIPEND-
82.5% (current default)
CLIPSTART
CLIPEND
94%
#CLIPSTART
#CLIPEND
92%
[CLIPSTART]
[CLIPEND]
96.5% (best)
\<CLIPSTART>
\<CLIPEND>
93.5%
\<clipboard>
\</clipboard>
87%
"Success rate" is measured by how often the AI correctly wraps information within the START_SEQ and END_SEQ tags when asked to copy to clipboard.
Results: [CLIPSTART] and [CLIPEND] is the most reliable style on Llama 3 8B. Consistently so over additional runs. The current defaults, with dashes (-), consistently perform the worst. I suspect dashes confuse the AI since they usually are used for bulleted lists and combined words.
ILikeAI
commented
3 weeks ago
Change the default config clipboard START_SEQ and END_SEQ to
"[CLIPSTART]"and"[CLIPEND]"for better reliability.To find this, I ran multiple styles through a benchmark of 20 different questions repeated 200 times for each style.
"Success rate" is measured by how often the AI correctly wraps information within the START_SEQ and END_SEQ tags when asked to copy to clipboard.
Results:
[CLIPSTART]and[CLIPEND]is the most reliable style on Llama 3 8B. Consistently so over additional runs. The current defaults, with dashes (-), consistently perform the worst. I suspect dashes confuse the AI since they usually are used for bulleted lists and combined words.