Prompt Quill was created to help users make better prompts for creating AI images.
Even if you are an expert, it could still be used to inspire other prompts.
It is useful for poor prompt engineers like me who struggle with coming up with all the detailed instructions that are needed to create beautiful images using models like Stable Diffusion or other image generators.
The world's first RAG driven prompt engineer helper at this large scale. Use it with more than 3.2 million prompts in the vector store. This number will keep growing as I plan to release ever-growing vector stores when they are available.
The Gradio UI will also help you to create more sophisticated text to image prompts.
Here you find just the code to run the UI or to insert data into your instance of the vector store.
There is no hardcoded need to use Docker at all, I understand this scares a lot of people. Docker is only here as it is a convenient way to run the vector stores, but it is easy to run them without docker too. Prompt Quill itself does not need Docker to run.
With the latest faeture added it also adds a way for automated model testing if you are into training models.
In the Character tab you will find "the magic prompt", which is the history that gets set when you enter your prompt to generate a new one.
Here you will get full control, and you can make it as close as possible to your personal prompting style.
Right now there are just a few examples, but if you change the Query and Answer to your type of prompts, it will generate prompts corresponding to your preferred style. You might need to experiment with this but during development, I learned that with this you can get a powerful tool to make it generate prompts as close as possible to your style.
There is now 2 different templates that will show how much difference you can get by changing the character. template_a tends to create story telling prompts, while template_b will create lists of concepts as most of SD prompters do.
This is the prompt rocket man in space
Here we can see the image the prompt from Prompt Quill created out of rocket man in space.
The full prompt from Prompt Quill was:
High-quality digital art, ultra-detailed, professional, clear, high contrast, high saturation, vivid deep blacks, crystal clear, ((rocket man in space)), wearing a full helmet and leather jacket, leather gloves, standing in front of an advanced high-tech space rocket, surrounded by the vastness of outer space, with intense, vibrant colors, colorful, dark, modern art style, the rocket illuminated by the cosmic light, the rocketman standing solo against the cosmic backdrop, bokeh effect creating a blurry background, photography-style composition, on eye level, masterpiece.
Looking at this prompt, keep in mind that you get full control about the prompt style by editing the history in the character tab.
You might wonder where the rocket is from the prompt. It is clearly not in the image and maybe it is hiding behind the guy in front. The outcome depends on the model you use and on how many samples you take using different seeds. The example you see here is the one I liked the most from the few samples I took. Also, the image dimensions used will have a large impact on what gets shown. A wide image will look totally different than a high image. (But that's AI image generation theories which are not part of this here.) Here, it's all just about making nicer prompts. It's up to you to create the right image setup for your prompts. Here are a few more samples using a different model.
Here you see images comparing a regular, hand made prompt versus a prompt made by Prompt Quill.
The left is the Prompt Quill prompt and the right is the hand made prompt, they use the same settings and seed.
Also they use a very detailed handcrafted negative prompt, the right image is the nicer one.
Now we see same prompts, same seed but with no negative prompt, and we can see how much the right one drifts away from the first one while the left drifts not as hard.
"sail to italy" would give you a prompt like this:
Stunning aerial photograph of a sailboat gracefully navigating through the crystal-clear waters of the Mediterranean Sea, en route to Italy. The sun casts a warm golden glow on the boat, while the distant villages and medieval towns along the coastline appear as intricate, detailed miniatures in the background. The image captures the beauty of the Italian coastline, with its picturesque beaches, islands, and castles, all bathed in the warm, golden light of the setting sun. The sailboat's sails billow with the wind, creating a sense of movement and adventure in this breathtaking scene. The photograph is taken in 8K resolution, offering an incredibly detailed and immersive view of the scene, with the stars twinkling in the night sky above, creating an Aurora-style atmosphere that adds to the magic of the moment.
Here are a few samples from 8 different free online image generators: We see the prompts are not just working for stable diffusion models.
Based on the information I retrieved for all those prompts data I also know the negative prompt that was used for a prompt and also I know what model was used to generate the image I took the prompt from, So why not give that information to you too. I managed to get this working in llama-index and haystack, the data is available on civitai.
Here you see an example "a nice cat" with no negative prompt on the left and on the right same prompt and seed but with negative prompt
I added the free online image generations from civitai and hordeai to allow you to get a first idea of where the prompt might go once you run it inside your image generation tool. Its easy to use but please be aware that your local results might be much nicer than what you get from that online things. In both cases the number of models you can use is quite limited. But still it gives you a first idea of how it might go.
To get data into your prompt quill there are two ways: the hard and the easy one ;)
The hard way is to go and get a large number of prompts and put them into a vector store. The scripts to do so are included.
For the easy way, you just run the one click install he will do all you need to run your prompts The data can be found here also: https://civitai.com/models/330412
If you like this project, and you are willing and able to sponsor this project as a longterm host (this including the data), feel free to contact me.
If you like and can provide large numbers of prompts please get in contact with me. I am compiling a growing vector DB which I can then share at some place where I can upload those files.
If you are on windows, you can use the one_click_install.bat. Since this is downloading very large files it can happen that downloads fail. You can use prepare_download_cache.bat to download the files and store them in a cache folder. The one_click_install.bat will check if the files are there and does not download them.
There is nothing preventing you from prompt injection or changing the way the LLM will behave. This is not the focus of this project as that is a whole different story to solve.
This will become a topic once there is a hosting provider for the project.
Please find me and join us on discord: https://discord.gg/gMDTAwfQAP