Feature preview: ai-generated image keywords

[chrisrude]

off by default, turn on with the new yaml setting: use_ai_generated_keywords

FEATURE PREVIEW: If set, the bot will ask Oobabooga to generate image keywords from a user's message. It will then pass the keywords that Oobabooga produces to Stable Diffusion to finally generate an image.

Otherwise, the bot will simply extract keywords directly from the user's message using a simple regex.

This is an experiment to see if we can get a more accurate image by passing the user's request to oobabooga first, to have it generate image keywords, rather than passing the request straight to stable diffusion.

To come: also include the bot's initial chat response to the user in the keyword generation. That way if the message tells the user that the bot is generating an image of something, that information is actually conveyed to Stable Diffusion.

Would welcome any comments on the advantages / disadvantages of this in your testing.

[chrisrude]

See https://github.com/chrisrude/oobabot/commit/92f0299b1e8a59ccfd98c5dc6fcfe9a07d13e430 for details

One thing that's cool is that this is an actual benefit of integrating images and text processing together, rather than just having a SD image generator inside a chat bot.

[chrisrude]

Having played with a few newer models, I've observed this works really well on some, and terribly on others. More work needed before I'd consider it "official".

[sidonsoft]

AI, please take the user's input and transform it into a rich, imaginative narrative. Enrich the description with vivid language, evoking a sense of wonder, magic, and beauty. Create a flowing narrative filled with striking details, breathtaking scenes, and unforgettable characters. Then, format this narrative into a list of descriptive phrases separated by commas. These descriptions should be concise yet powerful, suitable for guiding the creation of a complex and intricate image using an image generation tool. Remember to capture the nuances of the narrative and preserve the essence of the story in each phrase, allowing for a seamless visual interpretation

Imagine you are a master wordsmith, tasked with creating an imaginative narrative inspired by the user's prompt. You will be provided with a prompt from the user, and your goal is to craft a visually captivating story using rich, descriptive language. Your narrative should be filled with vivid imagery, transporting the reader to a world of wonder and enchantment. Remember to utilize a variety of sensory details, employing a palette of descriptive words to engage the reader's imagination. Each descriptive phrase should be separated by commas to facilitate its use in an image generation prompt. Take the user's prompt and let your creativity flow, weaving a tapestry of words that will bring the reader's imagination to life.

I have the results of various testing with image generation, if you want to have a browse, drop me a line

[Timboman]

I have oobabot setup via the oobabot_plugin on my docker instance of the Oobabooga TextGeneration-WebUI, whenever I try to add the "use_ai_generated_keywords: true" value to my config.yml file it seems to force delete it whenever the plugin initializes the bot. How exactly can I enable this. Is there an Environmental Value I can pass like with the Discord API key?

[jmoney7823956789378]

I have oobabot setup via the oobabot_plugin on my docker instance of the Oobabooga TextGeneration-WebUI, whenever I try to add the "use_ai_generated_keywords: true" value to my config.yml file it seems to force delete it whenever the plugin initializes the bot. How exactly can I enable this. Is there an Environmental Value I can pass like with the Discord API key?

The feature was removed :( Might have to check old commits and merge it back into your own fork.