This PR introduces enhancements to the benchmarking and report generation functionalities in the GuideLLM project. Key updates include the integration of detailed progress tracking using the rich library, improvements to performance statistics, and adjustments to report output formats.
Details
Added BenchmarkReportProgress class to manage real-time progress updates for benchmarks and report generation using rich.
Updated the report structure to include comprehensive performance statistics, such as request latency, time to first token, and inter-token latency.
Introduced new methods to generate detailed tables summarizing requests, data tokens, and performance metrics in reports.
Enhanced the executor logic to handle different benchmark modes (e.g., sweep, synchronous, throughput) more effectively.
Refined error handling and logging for better clarity and debugging.
Modified unit tests to cover new functionalities and ensure robustness across various scenarios.
Adjusted configuration and settings to optimize benchmarking capabilities and ensure consistent output.
Test Plan
Added new unit tests to verify the functionality of BenchmarkReportProgress and other report generation methods.
Performed end-to-end testing of different benchmark modes to ensure accurate real-time progress tracking and report generation.
Verified that all existing tests pass without any regression failures.
Summary
This PR introduces enhancements to the benchmarking and report generation functionalities in the GuideLLM project. Key updates include the integration of detailed progress tracking using the rich library, improvements to performance statistics, and adjustments to report output formats.
Details
BenchmarkReportProgressclass to manage real-time progress updates for benchmarks and report generation usingrich.Test Plan
BenchmarkReportProgressand other report generation methods.