robotology / robot-testing-framework

Robot Testing Framework (RTF)
http://robotology.github.io/robot-testing-framework/index.html
GNU Lesser General Public License v2.1
19 stars 11 forks source link

[testrunner] fail testrunner execution (i.e. return value different from zero) if the fixture manager fails #29

Closed traversaro closed 9 years ago

traversaro commented 9 years ago

In this Travis Job https://travis-ci.org/robotology/gazebo-yarp-plugins/jobs/62666628 :

testrunner --verbose --suit ../../icub-tests/suits/basics-icubGazeboSim.xml
Loading ../../icub-tests/suits/basics-icubGazeboSim.xml
[testrunner] cannot load plugin FtSensorTest.so; error (load) : FtSensorTest.so: cannot open shared object file: No such file or directory
[testrunner] cannot load plugin FtSensorTest.so; error (load) : FtSensorTest.so: cannot open shared object file: No such file or directory
[testrunner] cannot load plugin FtSensorTest.so; error (load) : FtSensorTest.so: cannot open shared object file: No such file or directory
Staring test runner.
Test suit Basic Tests Suite started...
[INFO]  (Basic Tests Suite) reports: yarpmanager is setuping the fixture...
||| did not find fixtures/icubgazebosim-fixture.xml
[ERROR] (Basic Tests Suite) asserts error on (appfile.size()) with exception: yarpmanager cannot find apllication file icubgazebosim-fixture.xml. Is it in the 'fixtures' folder?
[INFO]  (Basic Tests Suite) reports: yarpmanager is tearing down the fixture...
Test suit Basic Tests Suite failed!
Ending test runner.
-------- results ---------
Total number of tests : 0
Number of passed tests: 0
Number of failed tests: 0

The command "testrunner --verbose --suit ../../icub-tests/suits/basics-icubGazeboSim.xml" exited with 0.

The testrunner is returning 0 because the number of failed tests is zero, but we should make him aware of the ERROR in [ERROR] (Basic Tests Suite) asserts error on (appfile.size()) with exception: yarpmanager cannot find apllication file icubgazebosim-fixture.xml. Is it in the 'fixtures' folder? and return a value different from 0.

apaikan commented 9 years ago

Fixed in commit 35bcaf4e1bb211bff4928a1d6de5e7df4ff83b55