0x000011b
commented
1 year ago My guess on where this COULD be coming from is people dumping chat logs of cai and since the cai bots are pretty repetitive sometimes I guess that could be the isuse?
Possibly. I do some filtering on the submitted data to try and remove those kinds of repetitions but it's not perfect, so there's a portion of that still making it through and being used to train the models unfortunately.
However, that kind of exaggerated repetition within the same message sounds unusual. Perhaps try using the default generation settings but with the repetition penalty slightly increased (~1.2)? Or if there's very few example chats or persona data, you could try filling those in with more varied stuff.
The bot has been repeating words multiple times during a sentence, and just is repetetive in general, for example I'll get a sentence like "She laughed again, shaking her head. She replied, her voice dripping with sarcasm. She replied, her voice dripping with sarcasm. She replied, her voice dripping with scorn. She replied, her voice dripping with disdain." all in the same sentence. As well as the bot saying the exact same thing multiple times in the sentence, like saying "You're mine" 3 times in a row. My guess on where this COULD be coming from is people dumping chat logs of cai and since the cai bots are pretty repetitive sometimes I guess that could be the isuse?