We should have some public notification about the results of scenario tests without the need of logging in to TIER IV's evaluator console.
Background
Currently, we are doing weekly scenario tests within ODD WG using TIER IV's evaluator functions, and the results are visible under TIER IV's evaluator console. However, you need to create an account and ask admins add give them access to view the results. We should make the result of the weekly scenario tests publicly visible and reported to Autoware GitHub so that developers could know any regression quicker.
Possible Approach
There is WebAutoCLI which can retrieve the results from command line. We can have AWS host which use these commands that retrieves the result of scenario tests periodically and publish the result as dashboard.
We could also create an issue in Autoware GitHub for failing issue
Description
We should have some public notification about the results of scenario tests without the need of logging in to TIER IV's evaluator console.
Background
Currently, we are doing weekly scenario tests within ODD WG using TIER IV's evaluator functions, and the results are visible under TIER IV's evaluator console. However, you need to create an account and ask admins add give them access to view the results. We should make the result of the weekly scenario tests publicly visible and reported to Autoware GitHub so that developers could know any regression quicker.
Possible Approach
There is WebAutoCLI which can retrieve the results from command line. We can have AWS host which use these commands that retrieves the result of scenario tests periodically and publish the result as dashboard. We could also create an issue in Autoware GitHub for failing issue