I'm running MIPROv2 using v2.4.14 and v2.4.16 showing the same thing when running (no problem getting stuck), I'm wondering if this is something to watch for as I'm not sure if it is okay to ignore it:
WARNING: Projected Language Model (LM) Calls
Please be advised that based on the parameters you have set, the maximum number of LM calls is projected as follows:
- Prompt Model: 10 data summarizer calls + 4 * 2 lm calls in program + (3) lm calls in program aware proposer = 21 prompt model calls
- Task Model: 25 examples in minibatch * 10 batches + 20 examples in train set * 1 full evals = 270 task model calls
Estimated Cost Calculation:
Total Cost = (Number of calls to task model * (Avg Input Token Length per Call * Task Model Price per Input Token + Avg Output Token Length per Call * Task Model Price per Output Token)
+ (Number of calls to prompt model * (Avg Input Token Length per Call * Task Prompt Price per Input Token + Avg Output Token Length per Call * Prompt Model Price per Output Token).
For a preliminary estimate of potential costs, we recommend you perform your own calculations based on the task
and prompt models you intend to use. If the projected costs exceed your budget or expectations, you may consider:
- Reducing the number of trials (`num_batches`), the size of the trainset, or the number of LM calls in your program.
- Using a cheaper task model to optimize the prompt.
Error getting source code: unhashable type: 'list'.
Running without program aware proposer.
b: 10
summary: Prediction(
summary="The dataset consists primarily of news articles from CNN, covering diverse topics such as sports, politics, health, and social issues, with a strong emphasis on entity-rich content and complex relationships. Temporal and geographical contexts are frequently included, and the articles are concise and informative, reflecting CNN's journalistic style."
)
DATA SUMMARY: The dataset consists primarily of news articles from CNN, covering diverse topics such as sports, politics, health, and social issues, with a strong emphasis on entity-rich content and complex relationships. Temporal and geographical contexts are frequently included, and the articles are concise and informative, reflecting CNN's journalistic style.
0%| | 0/20 [00:00<?, ?it/s]
Seems to be coming from the dspy.Module, evaluation doesn't raise any issue but will fail with MIPROv2 once examples have been augmented. Closing this issue.
I'm running MIPROv2 using v2.4.14 and v2.4.16 showing the same thing when running (no problem getting stuck), I'm wondering if this is something to watch for as I'm not sure if it is okay to ignore it:
Sample Code
Logs: