"Correctness" of Orthogonally Derived Approaches

[samm82]

@smiths mentioned in #96 that "the prevalence of [orthogonally derived approaches] … in the documentation is another example of a problem with the testing literature." However, I think I disagree. I think that these labels are useful, as "black-box integration testing", for example, articulates both the technique (black-box) and the level (integration) of testing performed as a simple atomic unit. If someone were writing a paper on this topic or implementing black-box integration testing but not white-box integration testing, this term would provide clarity.

The point I attempt to make here is not that these are discrepancies, but that they are simply "labels" that combine existing information and are as such omitted from our scope. Since I track black-box testing and integration testing in my glossary, also tracking black-box integration testing isn't useful.

I also had the thought that if the categories of approaches (see #21) are truly orthogonal, then one approach from each category can be combined to give a clearer picture of a specific test approach being discussed/implemented, such as something like manual black-box integration performance testing (practice, technique, level, type). This might not be useful and depends on the "correctness" of these categorizations, so I don't include it in my thesis at this point.

[smiths]

If the original sources are using the "orthogonality" of the testing approaches as you describe to build testing approaches, I'm all for it. In that case I agree with what you have said above. I thought the original sources were defining black box integration testing as a "new" testing technique. That is what I would object to. It isn't a new technique, but the combination of two existing techniques.

[samm82]

I think there's some overlap between the two. For example, (in)formal reviews are a combination of (in)formal testing and reviews; the definitions given by the ISTQB glossary essentially just define them as "reviews that (do not) follow a formal process" with no specialized information, which could be them implying orthogonality or just inventing a new term. I'm not sure if it would be worth it to revisit these examples and see how these combinations are used if we just decide that they are out of scope…

[smiths]

Having these examples out of scope is fine. I think you should include some of the above discussion about how these kind of composite terms do come up in the standards. You can say how they are usually treated as a combination of orthogonal test approaches, but that sometimes they are treated like a new term (as you explained above). Either way, it is appropriate to have them out of scope.