PX4 / PX4-Autopilot

PX4 Autopilot Software
https://px4.io
BSD 3-Clause "New" or "Revised" License
8.48k stars 13.5k forks source link

Travis CI: Rewrite unit tests in G test framework #1526

Closed LorenzMeier closed 9 years ago

LorenzMeier commented 9 years ago

The existing unit tests (Firmware/unittests) are written without any test framework and lack proper return value checking (they are in fact test harnesses). Moving them to G test would be a great first step, and then adding more checks.

dagar commented 9 years ago

I'll start looking at this.

LorenzMeier commented 9 years ago

@dagar Thanks! Have you noticed the chat at http://gitter.im/PX4/Firmware ? Just in case any questions turn up. I don't have very good recommendations to give, essentially we just need a better version of the 'make tests' command, which runs the Makefile in the unit test directory. If CMake is an absolutely required and huge improvement for the test build system we can probably life with it becoming a requirement. Otherwise plain Makefiles are preferred. Going to test classes in the G Test framework also might not require one app per test, and so the build system is not any more a concern.

dagar commented 9 years ago

Has anyone started moving the existing tests yet? If not I'll start at the beginning with autodeclination_test.cpp.

LorenzMeier commented 9 years ago

Not yet, so sounds like a good idea!

LorenzMeier commented 9 years ago

@dagar Here are the current math tests: https://github.com/PX4/Firmware/blob/master/src/systemcmds/tests/test_mathlib.cpp

The quick fix would be to add + -, etc. What do you think about the architecture? Right now most tests are bundled in the tests app. Does that make sense, or should we split them up to to unit tests? I've documented the different test systems here: https://pixhawk.org/dev/unit_tests

LorenzMeier commented 9 years ago

@jgoppert Would you be willing to lend a hand with creating an automated HIL script based on your HIL work (https://github.com/PX4/HIL)? The overall setup I have in mind is similar to the auto test of APM (http://autotest.diydrones.com), however run on the actual hardware. We do have hardware connected to a test system which executes on pushes and PRs, so its actually reasonably easy to set up if you can run it locally fine. The auto test code might serve as inspiration, I would appreciate if we could retain clarity about licenses though (if you pull in code, we need to license our own test system as GPL). https://github.com/diydrones/ardupilot/tree/master/Tools/autotest

jgoppert commented 9 years ago

I'll be glad to work on this. I would like to move away from JSBSim to a native python FDM to make is more x-platform. Having something similar to APM auto-test would be very valuable. This was the basis of my existing PX4/HIL work.

I think at some point using uORB as a basis for a simple C++ SIL would also be possible. I started work on a python SIL using uORB but haven't finished it yet. I know there is also work on a uORB based ROS SIL but I haven't looked at this closely yet.

LorenzMeier commented 9 years ago

@jgoppert The hardware test is now up and running, so all infrastructure would be in place. We can now extend this to more sensor / unit tests and HIL.

@dagar If you want to / need to have some of the unit tests on HW you will now get auto-feedback as well, similar to Travis.

schnappschuss 2015-01-11 16 26 06

dagar commented 9 years ago

That's awesome. To start with I'll make sure the existing tests will actually cause a failure if something goes wrong.

LorenzMeier commented 9 years ago

Done on master