[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.
Description
This issue aims to provide comprehensive guidance for developers interested in contributing to Autoware by identifying, troubleshooting, and resolving issues within various test scenarios.
It will serve as a resource for planning developers looking for jobs, new-comers evaluating Autoware's test success, and contributors aiming to fix broken scenarios without causing regressions.
Purpose
The purpose of this documentation is to streamline the process for developers to engage with Autoware's testing framework effectively.
It will enable contributors to easily find scenarios, understand their results, reproduce issues, and confidently submit fixes ensuring no disruption to other parts of the software.
Target Audience
Planning developers seeking opportunities in new developments but unsure where to find relevant jobs.
New-comers interested in understanding the success rates and test situations in Autoware.
Developers who encounter a broken scenario and want to fix it but don't know where to start.
Possible approaches
Scenario Discovery: Where are the scenarios located, how do I find and see them?
Success Evaluation: How do I know which scenarios are failed, which has passed?
Failure Analysis: Why did the scenario fail? What metric was not achieved?
Environment Setup: How do I reproduce the failing scenarios on my own machine? Including all commit-sha's from scenario.
Regression Testing: If I fix a scenario, how do I ensure it doesn't break the rest? How to test all scenarios on my own machine and make sure things didn't get worse?
Definition of done
[x] Comprehensive guidelines on where and how to find scenarios are documented.
[x] Criteria for evaluating the success of a scenario are clearly defined.
[x] Instructions on identifying and analyzing the reasons for scenario failures are provided.
[x] Steps to reproduce the exact scenario environment are detailed.
[ ] Procedures for running regression tests to ensure no disruptions are described.
Checklist
Description
This issue aims to provide comprehensive guidance for developers interested in contributing to Autoware by identifying, troubleshooting, and resolving issues within various test scenarios.
It will serve as a resource for planning developers looking for jobs, new-comers evaluating Autoware's test success, and contributors aiming to fix broken scenarios without causing regressions.
Purpose
The purpose of this documentation is to streamline the process for developers to engage with Autoware's testing framework effectively.
It will enable contributors to easily find scenarios, understand their results, reproduce issues, and confidently submit fixes ensuring no disruption to other parts of the software.
Target Audience
Possible approaches
Definition of done