Open detlefm opened 1 month ago
This question has no simple answer. There are currently many people working on automatic prompt optimization. There are methods like textgrad, trace, dpsy, godel's agent etc etc. All of them claim that in some benchmark they do better. I am pretty sure what you used, has a similar mechanism. Now which mechanism is better, I have no idea. That is why I started this experiment to begin with. To see if textgrad, will improve the prompt. And to be honest this was more interesting to me than making a podcast app. There douzens like that.
These are the votes, I copied the one with the 8 votes. Probably a better test would be to stop getting feedback and then just vote until you get a reasonable score. : {"original": 2, "20241001_072128": 9, "20241002_103912": 1, "20241002_205113": 2, "20241003_164149": 8, "20241001_074114": 1, "20241005_164619": 5, "20241001_164128": 1, "20241005_231709": 4, "20241006_133003": 4, "20241008_072622": 1, "20241008_075210": 1, "20241008_081026": 2, "20241008_083354": 4, "20241008_091332": 6, "20241008_082517": 2, "20241008_110831": 1, "20241002_080247": 1, "20241008_115527": 4, "20241008_085508": 2, "20241008_122055": 3, "20241008_133116": 1, "20241008_140054": 1, "20241008_143550": 2, "20241008_152447": 1, "20241008_182838": 2, "20241008_191635": 4, "20241008_200450": 2, "20241008_113246": 1, "20241008_222928": 1, "20241009_003523": 1, "20241009_041538": 2, "20241009_162059": 1, "20241009_165939": 1} .
I think no matter the system prompt, to get a proper podcast, you will neeed to have more agents to split the text. For some reason most of my prompts give 3 minutes podcast...and this is too little for a paper of 20-30 pages. Btw if you want you can find me in discord with the same name artnoage
I haven't yet looked into the logic of "optimize_prompt".
But I noticed that in the enhancer_prompt.txt it starts with "You are a creative scriptwriter."
In the prompt history, all enhancer_prompt_xxxx.txt starting with "You are a creative enhancer."
I have no idea what that does to the AI or whether it makes any difference at all. I was just wondering where this difference comes from. Btw - this is the enhancer prompt suggestion from ChatGPT beta system instructions generator: