CommandLineToRunASingleTest

[mrT23]

PR Type

enhancement, configuration changes

---

Description

• Enhanced the CoverAgent to adapt test commands for running a single test using AI.
• Introduced a new function in PromptBuilder to handle AI-based command adaptation.
• Added a command line argument --run-each-test-separately to main.py for executing tests individually.
• Updated config_loader.py to include a new configuration file for AI-based command adaptation.
• Created a new TOML configuration file to define prompts for AI model interaction.

---

Changes walkthrough 📝

	Relevant files
Enhancement	CoverAgent.py Enhance test command parsing for single test execution      cover_agent/CoverAgent.py Added logic to adapt test command for a single test. Integrated AI-based function for command adaptation. Updated test command handling with new logic. +9/-5      PromptBuilder.py Implement AI-based test command adaptation function            cover_agent/PromptBuilder.py Added function to adapt test command using AI. Utilized AICaller for AI model interaction. Implemented error handling for command adaptation. +25/-0
Configuration changes	main.py Add command line argument for separate test execution        cover_agent/main.py - Added new command line argument for running tests separately. +6/-0      config_loader.py Update configuration loader with new settings file              cover_agent/settings/config_loader.py - Included new configuration file for AI-based command adaptation. +1/-0      adapt_test_command_for_a_single_test_via_ai.toml Add configuration for AI-based command adaptation                cover_agent/settings/adapt_test_command_for_a_single_test_via_ai.toml Added configuration for AI-based test command adaptation. Defined prompts for AI model interaction. +57/-0

---

💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

[codiumai-pr-agent-pro[bot]]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 Security concerns Command Injection: The code constructs command lines using string formatting with user-provided input (test_file_relative_path). While the path comes from internal sources, it's recommended to sanitize or escape the path to prevent potential command injection vulnerabilities.

⚡ Recommended focus areas for review Error Handling The command adaptation logic lacks proper error handling for cases where command parsing fails or returns invalid results Exception Handling The broad exception catch in adapt_test_command_for_a_single_test_via_ai may hide specific issues that should be handled differently Type Validation The run_each_test_separately argument uses bool type which may not handle CLI input correctly - should consider action='store_true' instead

[codiumai-pr-agent-pro[bot]]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Score
Possible issue	Fix command line argument type to properly handle boolean flags ___ **Change the type of --run-each-test-separately argument from bool to store_true action, as bool type will require an explicit value and may cause parsing issues.** [cover_agent/main.py [123-128]](https://github.com/Codium-ai/cover-agent/pull/224/files#diff-610f77abc2025129e317ec1f98b3bb375b790b2197ed3f635331cca0a06fd999R123-R128) ```diff parser.add_argument( "--run-each-test-separately", - type=bool, + action="store_true", default=False, help="Run each test separately. Default: False" ) ``` - [ ] **Apply this suggestion** Suggestion importance[1-10]: 9 Why: Using 'type=bool' for command line arguments is a common mistake that can lead to confusing behavior. Changing to 'action="store_true"' is the correct way to handle boolean flags in argparse.	9
	✅ Add error handling for missing command line arguments in test command parsing ___ Suggestion Impact: The commit added error handling for the case when the '--' substring is not found in the pytest command, which aligns with the suggestion. code diff: ```diff + try: + ind1 = test_command.index('pytest') + ind2 = test_command[ind1:].index('--') + new_command_line = f"{test_command[:ind1]}pytest {test_file_relative_path} {test_command[ind1 + ind2:]}" + except ValueError: + print(f"Failed to adapt test command for running a single test: {test_command}") ``` ___ **Add error handling for the case when 'pytest' is found in the command but the '--' substring is not found, which would cause an IndexError.** [cover_agent/CoverAgent.py [72-75]](https://github.com/Codium-ai/cover-agent/pull/224/files#diff-dac868643df79ec6dfcf446359468a88ba6a6617bb8ffa0139757f0fbf5195b1R72-R75) ```diff if 'pytest' in test_command: ind1 = test_command.index('pytest') - ind2 = test_command[ind1:].index('--') - new_command_line = f"{test_command[:ind1]}pytest {test_file_relative_path} {test_command[ind1 + ind2:]}" + try: + ind2 = test_command[ind1:].index('--') + new_command_line = f"{test_command[:ind1]}pytest {test_file_relative_path} {test_command[ind1 + ind2:]}" + except ValueError: + new_command_line = f"{test_command[:ind1]}pytest {test_file_relative_path}" ``` - [ ] **Apply this suggestion** Suggestion importance[1-10]: 8 Why: The suggestion addresses a potential runtime error that could crash the application when the '--' substring is not found in pytest commands. This is a critical defensive programming improvement.	8
	Add validation for AI-generated command line output ___ **Add validation for the returned YAML command line to ensure it's not empty or None before returning it.** [cover_agent/PromptBuilder.py [222-224]](https://github.com/Codium-ai/cover-agent/pull/224/files#diff-1602223755506d62450bdc4cea9140b0a11ccf93f7cd00151970cd1d649786adR222-R224) ```diff response_yaml = load_yaml(response) -new_command_line = response_yaml["new_command_line"].strip() +new_command_line = response_yaml.get("new_command_line", "").strip() +if not new_command_line: + raise ValueError("AI returned empty or invalid command line") return new_command_line ``` - [ ] **Apply this suggestion** Suggestion importance[1-10]: 7 Why: The suggestion adds important validation to prevent potential issues with invalid AI responses, using a safer dictionary access method and explicit validation of the command line value.	7

💡 Need additional feedback ? start a PR chat

[EmbeddedDevops1]

/ask In a sentence or two summarize what this PR is doing.

[codiumai-pr-agent-pro[bot]]

Ask❓

In a sentence or two summarize what this PR is doing.

Answer:

This PR enhances the CoverAgent by introducing AI-based functionality to adapt test commands for running a single test, adds a new command line argument --run-each-test-separately to execute tests individually, and updates configuration files to support these changes.