Suggestion: History based generation

[TheInvisibleMage]

The Concept

A new mode wherein the past history of the conversation is summarized and used to inform the SD prompt. The process would be approximately as follows:

• When a prompt is input and the model produces the corresponding output, pull in a configurable amount of the chat history.
• Perform another request to the model, consisting of the chat history prefaced with "Please summarize the following:" or something similar.
• When the summary is received, do not display it to the user/add to visible chat. Instead, use the summary to generate the SD prompts, sticking it into the original prompt as is currently done.

In theory, this would allow the images to reflect the chat history, causing images to follow the flow of the conversation.

Potential Issues

• This would require two calls to the model for every one user input, which is less than ideal.
• The results would be heavily dependent on the language used by the model when summarizing.
• This process would not track the history/contents of images already generated by the model, only the text, potentially resulting in images showing varying styles and scenarios. This could be theoretically rectified, however, either by maintaining a log of prompts used to generate images previously in the history, or by leveraging something similar to the "send_pictures" extension to allow recognition of previous image elements.

[Trojaner]

Sounds good to me, will add to the TODO list but have it disabled by default as it will tremendously slow down generation speed. I think existing images can be replaced with the image description it already generated.

[MysticDaedra]

I think this is what I thought this extension did, create images based on text generation history. I'm assuming that is not the case? What would the benefit be to use this extension vs just using regular SD.Next?

[Trojaner]

This is now implemented with the new "tool" generation mode, see https://github.com/Trojaner/text-generation-webui-stable_diffusion/releases/tag/1.6.1