search

Ekumen-OS / Noah-bot-simulation-ros2

Gazebo-ROS2 based Noah bot simulation
BSD 3-Clause "New" or "Revised" License
5 stars 2 forks source link

Add integration test package and headless mode #13

Closed tulku closed 1 year ago

tulku commented 1 year ago

Including a simple robot does not drift test

To run the test start the simulation, and then run:

$ ros2 run noah_integration_tests noah_integration_tests_odometry_drift_test

You should see an output like this:

root@4baecc5911cd:/colcon_ws# ros2 run noah_integration_tests noah_integration_tests_odometry_drift_test
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from OdometryDriftTest
[ RUN      ] OdometryDriftTest.IsDriftAccetable
[INFO] [1673268422.869056054] [odometry_drift_test]: First odom message received!.
[INFO] [1673268422.918131082] [odometry_drift_test]: Currently accumulated drift: 0.000001
[INFO] [1673268422.965896321] [odometry_drift_test]: Currently accumulated drift: 0.000002
[INFO] [1673268423.016283535] [odometry_drift_test]: Currently accumulated drift: 0.000002
[INFO] [1673268423.353383486] [odometry_drift_test]: Currently accumulated drift: 0.000003
[INFO] [1673268423.353466533] [odometry_drift_test]: Currently accumulated drift: 0.000004
[INFO] [1673268423.353511458] [odometry_drift_test]: Currently accumulated drift: 0.000005
[INFO] [1673268423.353546554] [odometry_drift_test]: Currently accumulated drift: 0.000006
[INFO] [1673268423.353698260] [odometry_drift_test]: Currently accumulated drift: 0.000006
[INFO] [1673268423.353740149] [odometry_drift_test]: Currently accumulated drift: 0.000007
[INFO] [1673268423.366899479] [odometry_drift_test]: Currently accumulated drift: 0.000008
[INFO] [1673268423.420928907] [odometry_drift_test]: Currently accumulated drift: 0.000009
[INFO] [1673268423.471269894] [odometry_drift_test]: Currently accumulated drift: 0.000010
[INFO] [1673268423.522373472] [odometry_drift_test]: Currently accumulated drift: 0.000010
[INFO] [1673268423.569856206] [odometry_drift_test]: Currently accumulated drift: 0.000011
[INFO] [1673268423.619760290] [odometry_drift_test]: Currently accumulated drift: 0.000012
[INFO] [1673268423.671839748] [odometry_drift_test]: Currently accumulated drift: 0.000013
[INFO] [1673268423.718960189] [odometry_drift_test]: Currently accumulated drift: 0.000013
[INFO] [1673268423.770886849] [odometry_drift_test]: Currently accumulated drift: 0.000014
[       OK ] OdometryDriftTest.IsDriftAccetable (1012 ms)
[----------] 1 test from OdometryDriftTest (1012 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1024 ms total)
[  PASSED  ] 1 test.

If the simulation is not running, the test will fail as there are no odometry messages, and the drift is infinity:

[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from OdometryDriftTest
[ RUN      ] OdometryDriftTest.IsDriftAccetable
/colcon_ws/src/noah_integration_tests/test/odometry_drift_test.cpp:87: Failure
Expected: (accumulated_drift_) <= (1e-3), actual: inf vs 0.001
Accumulated drift too high.
[  FAILED  ] OdometryDriftTest.IsDriftAccetable (1015 ms)
[----------] 1 test from OdometryDriftTest (1015 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1015 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] OdometryDriftTest.IsDriftAccetable

 1 FAILED TEST

You can also run the test using colcon test:

root@Eskarina:/colcon_ws# colcon test --event-handlers console_direct+
Starting >>> noah_description
Finished <<< noah_description [0.04s]
Starting >>> noah_gazebo
Finished <<< noah_gazebo [0.06s]          
Starting >>> noah_integration_tests
UpdateCTestConfiguration  from :/colcon_ws/build/noah_integration_tests/CTestConfiguration.ini
Parse Config file:/colcon_ws/build/noah_integration_tests/CTestConfiguration.ini
   Site: buildkitsandbox
   Build name: (empty)
 Add coverage exclude regular expressions.
SetCTestConfiguration:CMakeCommand:/usr/bin/cmake
Create new tag: 20230110-0946 - Experimental
UpdateCTestConfiguration  from :/colcon_ws/build/noah_integration_tests/CTestConfiguration.ini
Parse Config file:/colcon_ws/build/noah_integration_tests/CTestConfiguration.ini
Test project /colcon_ws/build/noah_integration_tests
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: noah_integration_tests_odometry_drift_test

1: Test command: /usr/bin/python3 "-u" "/opt/ros/foxy/share/ament_cmake_test/cmake/run_test.py" "/colcon_ws/build/noah_integration_tests/test_results/noah_integration_tests/noah_integration_tests_odometry_drift_test.gtest.xml" "--package-name" "noah_integration_tests" "--output-file" "/colcon_ws/build/noah_integration_tests/ament_cmake_gtest/noah_integration_tests_odometry_drift_test.txt" "--command" "/colcon_ws/build/noah_integration_tests/noah_integration_tests_odometry_drift_test" "--gtest_output=xml:/colcon_ws/build/noah_integration_tests/test_results/noah_integration_tests/noah_integration_tests_odometry_drift_test.gtest.xml"
1: Test timeout computed to be: 60
1: -- run_test.py: invoking following command in '/colcon_ws/build/noah_integration_tests':
1:  - /colcon_ws/build/noah_integration_tests/noah_integration_tests_odometry_drift_test --gtest_output=xml:/colcon_ws/build/noah_integration_tests/test_results/noah_integration_tests/noah_integration_tests_odometry_drift_test.gtest.xml
1: [==========] Running 1 test from 1 test case.
1: [----------] Global test environment set-up.
1: [----------] 1 test from OdometryDriftTest
1: [ RUN      ] OdometryDriftTest.IsDriftAccetable
1: [INFO] [1673343960.820459652] [odometry_drift_test]: First odom message received!.
1: [INFO] [1673343960.879023392] [odometry_drift_test]: Currently accumulated drift: 0.000001
1: [INFO] [1673343960.927109838] [odometry_drift_test]: Currently accumulated drift: 0.000002
1: [INFO] [1673343960.979029327] [odometry_drift_test]: Currently accumulated drift: 0.000002
1: [INFO] [1673343961.024011861] [odometry_drift_test]: Currently accumulated drift: 0.000003
1: [INFO] [1673343961.073021826] [odometry_drift_test]: Currently accumulated drift: 0.000004
1: [INFO] [1673343961.124044380] [odometry_drift_test]: Currently accumulated drift: 0.000005
1: [INFO] [1673343961.177766612] [odometry_drift_test]: Currently accumulated drift: 0.000005
1: [INFO] [1673343961.228301524] [odometry_drift_test]: Currently accumulated drift: 0.000006
1: [INFO] [1673343961.276068266] [odometry_drift_test]: Currently accumulated drift: 0.000007
1: [INFO] [1673343961.323874034] [odometry_drift_test]: Currently accumulated drift: 0.000008
1: [INFO] [1673343961.374286623] [odometry_drift_test]: Currently accumulated drift: 0.000009
1: [INFO] [1673343961.422902574] [odometry_drift_test]: Currently accumulated drift: 0.000009
1: [INFO] [1673343961.476271266] [odometry_drift_test]: Currently accumulated drift: 0.000010
1: [INFO] [1673343961.526372386] [odometry_drift_test]: Currently accumulated drift: 0.000011
1: [INFO] [1673343961.588337566] [odometry_drift_test]: Currently accumulated drift: 0.000012
1: [INFO] [1673343961.633424678] [odometry_drift_test]: Currently accumulated drift: 0.000012
1: [INFO] [1673343961.677475621] [odometry_drift_test]: Currently accumulated drift: 0.000013
1: [INFO] [1673343961.723505207] [odometry_drift_test]: Currently accumulated drift: 0.000014
1: [INFO] [1673343961.773711326] [odometry_drift_test]: Currently accumulated drift: 0.000015
1: [       OK ] OdometryDriftTest.IsDriftAccetable (1024 ms)
1: [----------] 1 test from OdometryDriftTest (1024 ms total)
1: 
1: [----------] Global test environment tear-down
1: [==========] 1 test from 1 test case ran. (1024 ms total)
1: [  PASSED  ] 1 test.
1: -- run_test.py: return code 0
1: -- run_test.py: inject classname prefix into gtest result file '/colcon_ws/build/noah_integration_tests/test_results/noah_integration_tests/noah_integration_tests_odometry_drift_test.gtest.xml'
1: -- run_test.py: verify result file '/colcon_ws/build/noah_integration_tests/test_results/noah_integration_tests/noah_integration_tests_odometry_drift_test.gtest.xml'
1/1 Test #1: noah_integration_tests_odometry_drift_test ...   Passed    1.16 sec

100% tests passed, 0 tests failed out of 1

Label Time Summary:
gtest    =   1.16 sec*proc (1 test)

Total Test time (real) =   1.16 sec
Finished <<< noah_integration_tests [1.22s]

Summary: 3 packages finished [1.57s]

(but you also need to run the simulation first)

This PR also adds the option to start the simulation in headless mode:

./docker/run ros2 launch noah_gazebo noah_gazebo.launch.py gazebo_gui:=false

Closes #9, #14

tulku commented 1 year ago

Regarding starting the simulation manually, we want to run this in robomaker, which starts the simulation and the robot code as two different things, so I wanted to accommodate that workflow. It would be nice to also gett launch_testing or colcon test integration. We could add that as a follow up issue.

tulku commented 1 year ago

Ok, now colcon test will run the test (but you need to start the simulation first)

tulku commented 1 year ago

@francocipollone I added a commit that closes issue #14 as the PR was not merged yet (and I cannot merge PRs)