search

iputils / iputils

The iputils package is set of small old utilities for Linux networking.
Other
641 stars 263 forks source link

Implement tests to cover basic functionality #243

Open pevik opened 4 years ago

pevik commented 4 years ago

Write posix shell test framework to cover basic functionality. For start I'd cover ping or arping, which are IMHO mostly used (and ping get most often regressions).

kerolasa commented 4 years ago

I am not sure what is meant with shell test framework. Taking what meson has to offer is likely the fitting way to add tests to this project.

https://mesonbuild.com/Unit-tests.html

MikeHigginbottom commented 4 years ago

I'd like to take this issue. Not sure that Meson's unit test support will be enough on its own though. It seems to assume you have some kind of test framework already in place (e.g. CUnit) in which the test executables are written. Meson would then call out to these exes as part of the build process. Would a combination of the two be appropriate?

pevik commented 4 years ago

@MikeHigginbottom sure, go ahead :). Unfortunately I'm too busy atm to work on it.

BTW my original idea was to have functional testing (e.g. all options work as expected; because that's what affect users). IMHO code is quite small (and it's not a library) therefore I don't see a priority for unit testing. But maybe I'm wrong.

For functional testing I planned to start with ping and arping. I intend to reuse shell part of LTP test framework (https://github.com/linux-test-project/ltp/blob/master/testcases/lib/tst_test.sh it would require to take few parts of C code from C API as well), with it's network extension (https://github.com/linux-test-project/ltp/blob/master/testcases/lib/tst_net.sh). This framework is well established, network namespaces testing requires zero setup from user (except root privileges for setting netns). Here is some general info about LTP shell API https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#23-writing-a-testcase-in-shell (unfortunately network extension hasn't been documented yet).

These are just my thoughts, which I planned to implement. But feel free to do something completely different.

pevik commented 4 years ago

We should also test code on kernel without IPv6 support. Although I expect IPv6 everywhere, there are obviously people running IPv4 only: https://github.com/iputils/iputils/issues/293

I don't have much time for a development, but consider tests as a high priority. Hope I'll find time to prepare subset of LTP shell API and write at least some tests soon.

pevik commented 3 years ago

I've started working on testsuite. Our testsuite should be probably based on meson test command run in builddir

I'm working on simplifying LTP interface to use it. I'm also considering whether meson unit tests would be enough. We might need unit tests, but I'd like to have regression tests and these will need to have more functionality e.g. compare the output than just running command and expect 0 exit. Other question is whether using C/shell LTP test framework when meson probably allows to write testing API in python.