ThereforeGames
commented
11 months ago Hi @aljungberg,
Thank you for reaching out - that was an excellent writeup! Your observations as to why repetition_penalty does not address the underlying problem make a lot of sense. In my own tests, I found that at low values it's indistinguishable from placebo and at higher values it does more harm than good, for the reasons you outlined.
It would also be interesting to see if we can further improve the outcomes by excluding some bits from the negative because they are in fact representative of desirable repetition, like the prefix \nASSISTANT: or whatever.
I implemented a Blacklist feature in v0.1.0 that will allow you to test this; it uses regex to replace each matched term or phrase in the negative prompt with an empty string.
By default, I think the character name is already excluded in chat mode, but it may be necessary to use the Blacklist for chat-instruct or Notebook operations.
Speaking of which, Notebook support is a great idea. Let me do some research on the WebUI's extension framework to see if we can add that.
That said, the reason I find it surprising that your method works well in your testing is that it is rather indiscriminate.
It surprised me too! At first, I was testing the idea manually by pasting a message a bunch of times into the negative prompt box. It wasn't long before I had a, "what the hell, this actually works?" moment. Had to make an extension for it.
Thanks again!
aljungberg
Cool idea! If this performs as effectively in broader applications as in your tests, that's an intriguing and non-obvious discovery[1]. I like it.
I've noticed an issue or possibly a missing feature: when using the notebook tab, the content doesn't seem to integrate with Echoproof. The negative prompt, as observed in the console with debug mode enabled, only reflects the chat history tab.
Considering that you need to know the last n messages, I suggest a configurable regex splitter (defaulting to \n) to parse notebook content. That would create an equivalent input stream, nothing else really needing changing.
[1]: Not to digress but here's my opinion both as to why your method is promising yet non-obvious.
There are LLM repetition issues which escape existing control measures basically entirely. Transformers work well because they find patterns of patterns. Overall this is desirable. If every other line starts with
ASSISTANT:that's a useful pattern to extend. A human would do the same.But LLMs can overdo it, get into a repetitive thematic loop beyond what a person would. Whereas human brains generally notice and remember what we have already said and recognise the diminishing returns of repeating the same idea ad nauseam, that higher order anti-pattern has so far turned out to be difficult for LLMs to internalise. Perhaps we don't train them on long enough context windows, maybe the transformers aren't deep enough for such high-order activation suppression, or maybe much of the scraped web we use for training material is in fact inhumanly repetitive.
This issue is exacerbated by a self-reinforcing feedback loop. The more the model attends to particular patterns, the more the generation history provides evidence that this kind of repetition is indeed desired in the current context. This is IMHO the core of the repetition degeneration problem. We often don't even see the patterns the model is latching onto until it's too late and they are well and truly baked in.
Repetition penalty doesn't help because it's not a problem of individual tokens recurring, but longer range patterns. There are many ways to be repetitive using new words. I believe this is why setting a very high repetition penalty can lead to even worse degenerative behaviour like when the model starts writing in all caps or without punctuation or in emojis. It's trying really hard to satisfy these long range patterns but it's running out of ways to express them. We have penalised individual tokens but not the wider patterns. It's like we're telling the model to talk about forests with an ever larger exclusion on synonymous ways to say "tree". We might as well penalise individual letters of the alphabet: it's way too blunt an instrument to target abstract ideas.
So negative prompting seems like a perfect fit. The negative prompt will light up those activations most related to the larger patterns the model is most "addicted" to at the moment, and CFG will cause a push-back proportional to the strength of those activations.
That said, the reason I find it surprising that your method works well in your testing is that it is rather indiscriminate. We're sticking the good and the bad into the negative prompt. There are some patterns we do want close continuous adherence to. Writing style, overall subject matter, heck even what language we're speaking. Like generally speaking we want the model to keep doing what it's doing in most ways, we are just looking to shut down instances of pathological repetition.
Perhaps the effectiveness of your method lies in achieving a balance between the positive CFG and the negative CFG. Like, even if the negative prompt is all in English, that doesn't mean the model must now switch to a different language because the back-pressure against that wider pattern (what language we're speaking) is more than countered by the positive CFG. It would be an interesting line of inquiry to examine how outcomes are affected by the proportion of messages used for negative prompting. It would also be interesting to see if we can further improve the outcomes by excluding some bits from the negative because they are in fact representative of desirable repetition, like the prefix
\nASSISTANT:or whatever.I look forward to seeing where this goes!