Add some unit tests to the debian packaging

MRPT / mrpt

:zap: The Mobile Robot Programming Toolkit (MRPT)

https://docs.mrpt.org/reference/latest/

BSD 3-Clause "New" or "Revised" License

1.94k stars 630 forks source link

Add some unit tests to the debian packaging #299

Open jolting opened 8 years ago

jolting commented 8 years ago

Debian has a continuous integration which can be added to the packaging: https://ci.debian.net/

I thought this might be useful for testing across various platforms: https://ci.debian.net/status/platform-specific-issues/

It might be useful to create tests using ctest: https://cmake.org/Wiki/CMake/Testing_With_CTest

And the tests can be invoked using autopkgtests: http://packaging.ubuntu.com/html/auto-pkg-test.html http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/tree/doc/README.package-tests.rst

jlblancoc commented 8 years ago

Thanks, another project I didn't know!

Anyway, at present all "regular" mrpt unit tests are already run in the Debian packages via "make test" (well... actually via the equivalent "dh_auto_build" command), and they are run in i386/amd64/armhf everytime the packages are manually pushed to Ubuntu PPA...

DebianCI would allow even faster and more automated testing, so it may be worth trying...

jolting commented 8 years ago

There are a few advantages of using debian ci that I see.

A uniform way to capture test reports for all debian packages.
Burdening the build system with tests is usually a bad idea. Unit tests may be offloaded to other machines.
The packaging of the binaries still succeed regardless of test failure. Whereas unit tests that cause builds to fail will not generate binaries.
If you're using your own jenkins server, the ability to turn off or defer the tests until a later time.

ctest is similar to what you are doing with make tests. In fact it uses that Makefile target, so it might conflict with your make tests. Each target is added as a test using add_test. The ability to post test results to cdash looks interesting though.

jlblancoc commented 8 years ago

mmm.. I like the cdash stuff, I'll check it out. Thanks!

I think I didn't get one of your point, though...

The packaging of the binaries still succeed regardless of test failure. Whereas unit tests that cause builds to fail will not generate binaries.

This already happens with the current approach. As you can see in the debian/rules file (basically a Makefile) there is one step at the end of building the .deb files where unit tests are invoked... and if one of the them fail, the .deb generation is aborted. Is this what you meant? It's only now that we are about to succeed in building mrpt, and passing all tests in all Debian platforms, which is worth celebrating!! ;-)

In fact, the burst of activity the latest days about reverseBytes*(), etc. was due to problems in unit tests in weird platforms (sparc64,mips,...)! But it's worth the effort, because at the end it turns out to be caused by some brute-force type casting which were not 100% valid...