Memory and world info editing

[ArkyMosuke]

KoboldAI allows editing of World Info and Memory through the web interface's main page, which can be used to enhance responses. Would it be possible to replicate this functionality in gradio-ui so they can be used?

https://github-wiki-see.page/m/KoboldAI/KoboldAI-Client/wiki/Memory%2C-Author%27s-Note-and-World-Info

[ArkyMosuke]

I'm so hoping to have this feature in the gradio API. Being able to anchor the AI in surroundings more firmly would be amazing. Here's hoping it comes easily enough.

[0x000011b]

I do have plans for external knowledge grounding (whether that be long-term memory stored somewhere, internet searches, or manually added world info), but that'll require new versions of the models to be trained and some research work on figuring out the ideal ways to select and inject the bits of knowledge into the prompt, so it's out of scope of the current prototype UI.

If you like Kobold's implementation though, the model is already usable there I believe. Henk pushed some fixes to chatmode so it theoretically supports the Pygmalion models, although I'm not sure whether that's on the official version or his United branch.

[ArkyMosuke]

I do have plans for external knowledge grounding (whether that be long-term memory stored somewhere, internet searches, or manually added world info), but that'll require new versions of the models to be trained and some research work on figuring out the ideal ways to select and inject the bits of knowledge into the prompt, so it's out of scope of the current prototype UI.

If you like Kobold's implementation though, the model is already usable there I believe. Henk pushed some fixes to chatmode so it theoretically supports the Pygmalion models, although I'm not sure whether that's on the official version or his United branch.

I've tried using it in koboldAI but it seems like the model doesn't use any of the world info or memory. Sadly.

[0x000011b]

Yep, very likely due to the way it was trained. I'll have to add some special tokens to the training data that teach the model to look at specific parts of the prompt when generating the response, and then the external knowledge grounding system will use those tokens when prompting the model with memory/world info/search results.

[ArkyMosuke]

Yep, very likely due to the way it was trained. I'll have to add some special tokens to the training data that teach the model to look at specific parts of the prompt when generating the response, and then the external knowledge grounding system will use those tokens when prompting the model with memory/world info/search results.

I would kill for a long term memory for this model. So I'll be waiting with bated breath. I'm considering buying 4-5 k60's or k80's to experiment with a larger model and see how it plays out. I've been intrigued by AI creative writing for a while now