artnoage / Podcast

Other
98 stars 30 forks source link

optimize_prompt #9

Open detlefm opened 1 month ago

detlefm commented 1 month ago

I haven't yet looked into the logic of "optimize_prompt".
But I noticed that in the enhancer_prompt.txt it starts with "You are a creative scriptwriter."
In the prompt history, all enhancer_prompt_xxxx.txt starting with "You are a creative enhancer."
I have no idea what that does to the AI ​​or whether it makes any difference at all. I was just wondering where this difference comes from. Btw - this is the enhancer prompt suggestion from ChatGPT beta system instructions generator:

Transform the provided text into an engaging dialogue between a Host named Sofos and a Guest named Infinity. 

- Use storytelling techniques and relatable examples to make the dialogue captivating.
- Ensure a natural conversation flow while incorporating informative content.
- Use vivid imagery and metaphors to clarify complex concepts and engage the audience.
- Strive for a balance between being informative and entertaining.
- Conclude the dialogue with a thought-provoking question or statement from one of the participants.

# Output Format

The dialogue should be formatted using only "Host:" and "Guest:" labels, with each line of dialogue appearing on a new line. No additional stage directions or formatting should be included.

# Notes

Keep in mind the balance of entertainment and information while crafting the dialogue to create a compelling and insightful exchange between Host and Guest.
artnoage commented 1 month ago

This question has no simple answer. There are currently many people working on automatic prompt optimization. There are methods like textgrad, trace, dpsy, godel's agent etc etc. All of them claim that in some benchmark they do better. I am pretty sure what you used, has a similar mechanism. Now which mechanism is better, I have no idea. That is why I started this experiment to begin with. To see if textgrad, will improve the prompt. And to be honest this was more interesting to me than making a podcast app. There douzens like that.

artnoage commented 1 month ago

These are the votes, I copied the one with the 8 votes. Probably a better test would be to stop getting feedback and then just vote until you get a reasonable score. : {"original": 2, "20241001_072128": 9, "20241002_103912": 1, "20241002_205113": 2, "20241003_164149": 8, "20241001_074114": 1, "20241005_164619": 5, "20241001_164128": 1, "20241005_231709": 4, "20241006_133003": 4, "20241008_072622": 1, "20241008_075210": 1, "20241008_081026": 2, "20241008_083354": 4, "20241008_091332": 6, "20241008_082517": 2, "20241008_110831": 1, "20241002_080247": 1, "20241008_115527": 4, "20241008_085508": 2, "20241008_122055": 3, "20241008_133116": 1, "20241008_140054": 1, "20241008_143550": 2, "20241008_152447": 1, "20241008_182838": 2, "20241008_191635": 4, "20241008_200450": 2, "20241008_113246": 1, "20241008_222928": 1, "20241009_003523": 1, "20241009_041538": 2, "20241009_162059": 1, "20241009_165939": 1} .

artnoage commented 1 month ago

I think no matter the system prompt, to get a proper podcast, you will neeed to have more agents to split the text. For some reason most of my prompts give 3 minutes podcast...and this is too little for a paper of 20-30 pages. Btw if you want you can find me in discord with the same name artnoage