Some question for code optimization

[Thewillman]

Well，as the batch api has published a few months ago， why not change single generation to batch generation？The cost will be half and the pipeline will be more efficient， maybe test the whole dataset can take less cost.

[ShengranHu]

Hello,

While the batch API is indeed valuable for non-iterative tasks such as dataset processing, it is less suitable for iterative algorithms like ADAS and Meta Agent Search due to its 24-hour turnaround time. This delay means that each iteration would require waiting a full day to continue optimization. We recommend users try the latest cost-efficient models, such as GPT-4o-mini, to help reduce costs.

[Thewillman]

I’m sorry for my unclear writings. For agent profile update and validation set evaluate, there is no need for batch api and gpt-4/4o maybe the suitable model for this stage. However, the test set evaluation stage is possible to use the batch api to accelerate and lower the cost

[Thewillman]

The motivation is interesting and may be helpful to some researchers who are troubled by handwriting agents, so I care for some details like cost and code scalability

[ShengranHu]

Oh, I see. I think it could be a good practice. Thank you for your suggestions!