BerkeleyLearnVerify / Scenic

A compiler and scenario generator for the Scenic scenario description language.
https://scenic-lang.org/
Other
258 stars 93 forks source link

feat: adding webots dynamic scenario tests #231

Open abanuelo opened 3 months ago

abanuelo commented 3 months ago

Description

Adding a webots dynamic scenario test where an object is created above a plane and then drops until it hits the floor. The test checks the expected start and ending z values for the object's position. Note, that webots runs headlessly via a python subprocess. This subprocess then creates a results.txt file which contains the starting and ending z positions. Below is an illustration of the test case running on our machine:

https://github.com/BerkeleyLearnVerify/Scenic/assets/32311654/dbade0af-69fc-48a9-b4c5-ea84c27d68ab

Issue Link

Checklist

Additional Notes

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.27%. Comparing base (d7679fb) to head (185fe3c).

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/BerkeleyLearnVerify/Scenic/pull/231/graphs/tree.svg?width=650&height=150&src=pr&token=HN3J4Y6F89&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BerkeleyLearnVerify)](https://app.codecov.io/gh/BerkeleyLearnVerify/Scenic/pull/231?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BerkeleyLearnVerify) ```diff @@ Coverage Diff @@ ## main #231 +/- ## ========================================== - Coverage 91.32% 91.27% -0.06% ========================================== Files 54 54 Lines 13515 13515 ========================================== - Hits 12343 12336 -7 - Misses 1172 1179 +7 ``` [see 3 files with indirect coverage changes](https://app.codecov.io/gh/BerkeleyLearnVerify/Scenic/pull/231/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BerkeleyLearnVerify)
abanuelo commented 3 months ago

Important!!!!

In order to for dynamic webots tests to work you need a physical or virtual display. If you are:

  1. sshing into the instance and triggering the webots tests manually
  2. OR running the CI #226 that triggers simulation tests on the instance

it is VERY important you create a virtual screen prior to running the tests!

Xvfb :99 -screen 0 1024x768x16 &
export DISPLAY=:99

If not you will generate the following warning which will automatically fail all webots dynamic scenario tests:

Warning: Could not find the Qt platform plugin "offscreen" in ""
Fatal: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb.

/usr/local/bin/webots: line 105: 12490 Aborted                 (core dumped) "$webots_home/bin/webots-bin" "$@"

If you are using the DCV client to run the test (as illustrated in the PR description above), the built in DISPLAY port should be fine and no need to run the commands above.