This PR refactors the benchmark report generation process and updates backend configurations for better performance and clarity. The main changes include restructuring the entry point for benchmark generation, consolidating backend configurations, and improving error handling for better user experience.
Details
Introduced generate_benchmark_report_cli as the main CLI entry point for benchmark generation.
Refactored generate_benchmark_report to handle various input configurations and streamline the execution process.
Removed unused parameters (host, port) and consolidated backend URL configurations into a single target parameter.
Updated OpenAIBackend initialization to eliminate ambiguity around base URL settings.
Enhanced error handling and logging, ensuring informative messages for missing or incorrect configurations.
Modified EmulatedRequestGenerator and FileRequestGenerator to handle optional configuration inputs.
Removed obsolete CLI test files and restructured e2e tests to align with the new CLI entry point.
Added comprehensive unit tests for the main benchmark generation functions and CLI commands.
Summary
This PR refactors the benchmark report generation process and updates backend configurations for better performance and clarity. The main changes include restructuring the entry point for benchmark generation, consolidating backend configurations, and improving error handling for better user experience.
Details
generate_benchmark_report_clias the main CLI entry point for benchmark generation.generate_benchmark_reportto handle various input configurations and streamline the execution process.host,port) and consolidated backend URL configurations into a singletargetparameter.OpenAIBackendinitialization to eliminate ambiguity around base URL settings.EmulatedRequestGeneratorandFileRequestGeneratorto handle optional configuration inputs.