search

zou-group / textgrad

TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.
http://textgrad.com/
MIT License
1.75k stars 148 forks source link

TextualGradientDescent optimizer response could not be indexed #83

Open Shad0wSeven opened 3 months ago

Shad0wSeven commented 3 months ago

I keep getting this error

ERROR: TextualGradientDescent optimizer response could not be indexed. This can happen if the optimizer model cannot follow the instructions. You can try using a stronger model, or somehow reducing the context of the optimization. Response: <VARIABLE> [INPUT] description: Input to the chatbot, which may include questions, requests for information, or clarifications. The input should be clear and concise to ensure accurate responses. [OUTPUT] response: <Your output here> Guidelines for responses: 1. Provide clear, concise, and accurate answers. 2. If the question is ambiguous or unclear, ask for clarification politely. 3. Use a friendly and professional tone. 4. Ensure the response is relevant to the input. 5. If the information is not available, state that clearly and offer alternative suggestions if possible. </VARIABLE>

whenever using longer prompts or differnet types of data, following prompt optimization, is ther any fix? Running GPT-4o

simra-shahid commented 3 months ago

You can try reducing the batch size.

In optimizer.py, the prompt_update_parameter in line 177 has long feedback for input prompts that are long or short. Even though the system prompt + prompt_update_parameter token length is below gpt-4 max tokens, this error comes up. However, reducing the batch size to 1 or 2 is working. Tested with evaluator engine = gpt-4o, gpt-4.

gl-donald commented 2 months ago

I am running into the same issue with both FormattedLLMCall's and BlackboxLLM. I am using very large variables for context 20000 tokens. reducing the context has gotten the optimizer to run but doing so makes my loss very high. I hope there's a way around this i'm looking to optimize only the prompt not the context.