Add support for Anthropic's Message Batches API for efficient code analysis

[bitdeep]

Issue

https://docs.anthropic.com/en/docs/build-with-claude/message-batches

Description:

• Anthropic recently introduced their Message Batches API, which offers a cost-effective way to process large volumes of queries asynchronously. This feature could significantly enhance Aider's capabilities for code analysis and reporting.
• We propose integrating this API to improve Aider's performance and cost-effectiveness for large-scale code processing tasks. Specially for documentation building, test-cases creation, etc.

Proposed Implementation:

• Implement batch processing for long running and costly tasks, like code documentation, test-cases creation.

Integrate the Message Batches API:

• Add support for the anthropic-beta: message-batches-2024-09-24 header in API requests.
• Implement batch processing for code analysis tasks.
• Enhance Code Analysis Process:
  • Modify Aider to break down large codebases into batches of up to 10,000 requests.
  • Submit these batches for asynchronous processing using the new API.
• Asynchronous Processing:
  • Implement a system to track and manage submitted batches.
  • Add functionality to poll for status updates on submitted batches.
• Result Aggregation:
  • Collect and aggregate results from processed batches.
  • Compile findings into a comprehensive report.
• Reporting:
  • Create a new REPORTS.md file in the project root.
  • Write all analysis results and reports to this file.
  • Include summaries, code quality metrics, and suggested improvements.
• User Interface:
  • Add commands to initiate batch analysis (e.g., /batch-analyze).
  • Provide status updates on batch processing progress in the running console.
• Benefits:
  • Cost Reduction: Utilize the 50% discount offered by the Batches API.
  • Increased Throughput: Process larger codebases more efficiently.
  • Comprehensive Analysis: Leverage asynch processing for more thorough code reviews.
• Considerations:
  • Implement error handling for batch processing failures.
  • Ensure proper management of API rate limits.
  • Provide options for users to customize batch sizes and processing intervals.

Version and model info

No response

[fry69]

Thank you for filing this issue.

I can't see how an interactive terminal application could benefit from an up to 24 hours delayed batch service.

[bitdeep]

I can't see how an interactive terminal application could benefit from an up to 24 hours delayed batch service.

My idea here is that aider start exploring more an agentic long time behavior, not sure if you know, but when you do a do-release-upgrade on ubunto server, it auto-setup an screen and ssh session for you when you do it remotely.

Or in a more complex service model, aider can auto-setup itself as systemd/windows service to run this agent services.

So, I think that aider can start experimenting with agentic behavior this way, hope you guys like the idea.

[potatoqualitee]

I'm working on an expensive project where I have more time than money. I am happy to wait 24 hours. Would love this feature as well and will likely implement it in PowerShell + aider if there will be no official support.