search

wilkie / djehuty

Fully verifiable operating system and API written in D in the public domain.
39 stars 6 forks source link

Mechanism for testing private behaviours #18

Open 0 opened 15 years ago

0 commented 15 years ago

Because what goes on in private can be so wrong and you might not even know it.

0 commented 15 years ago

Note to self: TuiTextBox LineInfo should be tested when this is implemented.

wilkie commented 15 years ago

The framework cannot be responsible for private behaviors of the packages, only that the public behaviors are consistent with package implementations. The tests are simply to allow others to re-implement packages or classes with the same interface and then have the same tests imposed upon them across the framework to see if they function.

Tests are done external to the class for a reason: you don't want to have to change the test when you update the implementation. Private behaviors are so incredibly tied to a class that it is always difficult to test them.

If correct behavior of LineInfo can be ascertained through the correct behavior of the TuiTextBox, then that is certainly fine. A test member function exposed through a class is also fine, but it probably shouldn't be exposed as 'public', but rather as 'package' and called from a separate test module within the package.

Overall, private behaviors tell you nothing about the certifiable state of the program. Assume correctness of private behaviors. Produce some code in TuiTextBox that also assumes correctness and produces some public behavior. Test this code instead.

0 commented 15 years ago

Produce some code in TuiTextBox that also assumes correctness and produces some public behavior. Test this code instead.

The resulting behaviour is interactive, so I can't think of an automated way to test it.

steveklabnik commented 15 years ago

When testing, you only care about external behaviors. If you tried to test private behavior, you'd be violating encapsulation, amongst other things. If you get the correct answer, how could internal stuff be 'wrong?'

0 commented 15 years ago

It could be correct for the cases that are used internally, but fail for cases which are not used internally now but may be in the future.

steveklabnik commented 15 years ago

So you're going to test for everything that might be possibly true in the future?

Good luck.

wilkie commented 15 years ago

Steve's right. You can only account for the cases you acknowledge. So...

It could be correct for the cases that are used internally, but fail for cases which are not used internally now but may be in the future.

In this case, we would add specifications to account for additional features (features in this context are ALWAYS public behaviors... so ignore "performance" features which are tested differently), and those additional tests should catch the error and help reduce the case for the cause.

Not catching the error in the public behavior is our fault. Not having a test that would cover internal behavior is OK if the flaw is never exercised. Sounds icky, but it is actually fiiiine. What you are really worried about is passing a bad piece of data to another class (where flawed data propagation is a factor of class cohesion properties). So that is what you test for.