This PR optimizes the evaluation process by applying batch processing and concurrency techniques to significantly improve performance. Incorporated p-limit for managing concurrency and refactored the evaluation functions to handle multiple requests more efficiently.
Key Changes:
Batch Processing: Refactored evaluation functions to handle multiple questions concurrently instead of sequentially.
Concurrency with p-limit: Installed and integrated p-limit to control the concurrency of API calls, preventing overload while still optimizing request throughput.
Improved Evaluation Flow: Optimized both the question evaluation and report generation steps to reduce latency and handle bulk data more effectively.
This PR optimizes the evaluation process by applying batch processing and concurrency techniques to significantly improve performance. Incorporated p-limit for managing concurrency and refactored the evaluation functions to handle multiple requests more efficiently.
Key Changes:
Batch Processing: Refactored evaluation functions to handle multiple questions concurrently instead of sequentially.
Concurrency with p-limit: Installed and integrated p-limit to control the concurrency of API calls, preventing overload while still optimizing request throughput.
Improved Evaluation Flow: Optimized both the question evaluation and report generation steps to reduce latency and handle bulk data more effectively.