feat: adding ci for simulator testing

BerkeleyLearnVerify / Scenic

A compiler and scenario generator for the Scenic scenario description language.

https://scenic-lang.org/

Other

258 stars 93 forks source link

feat: adding ci for simulator testing #226

Closed abanuelo closed 2 months ago

abanuelo commented 3 months ago

Description

Adding CI in place to be able to test dynamic scenarios for CARLA and Webots. This CI has been tested privately! In subsequent PRs, dynamic scenarios for CARLA will be built using PyTest. Documentation has also been updated internally for how to get access to the instance running simulator-based tests.

End to End Test Passing with Webots and CARLA dynamic tests changes present on the main branch (ie #227 #231)

Issue Link

Checklist

[X] I have tested the changes locally via pytest and/or other means
[X] I have added or updated relevant documentation
[ ] I have autoformatted the code with black and isort
[ ] I have added test cases (if applicable)

Additional Notes

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 86.12%. Comparing base (d5a21fc) to head (320b8ef). Report is 5 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #226 +/- ## ========================================== + Coverage 85.89% 86.12% +0.23% ========================================== Files 144 144 Lines 24767 24938 +171 ========================================== + Hits 21274 21479 +205 + Misses 3493 3459 -34 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

abanuelo commented 3 months ago

Note: in order for this workflow to become active in our Scenic repo we need to add all secrets to our repository! I do not have privileges to do so, but happy to collaborate with others to make sure this happens!