Remove timing based tests

[ayberkozgur]

I don't think timing based tests are a good idea; they are not reliable enough for unit testing. E.g integration fails regularly on my machine, about 50% of the time. This failure is not due to division by zero; it's just due to background load that happens to be there.

[qbonnard]

That's a bit severe towards timing based tests if they're done properly, which was not the case. The test measured that the time to process a natively small image was more or less the same than the time to process an equally reduced image. The threshold was too aggressive though, and the test not so important. Since https://github.com/chili-epfl/chilitags/commit/8ee0e10a74626a6b6e8af23ed340f619b38fcdcc the test only checks that processing with a maxwidth is faster than without. That's actually more useful as a unit test and should not be a problem anymore, since there is about 2 orders of magnitude between the two.

[ayberkozgur]

That's my point, there is no way of doing them 100% properly, which means with 100% repeatable results, without counting the instruction cycles in a controlled environment (callgrind style). To me, timers can be samples but not unit tests which require 100% repeatability by definition.

[qbonnard]

There is no way of doing anything 100% properly ;) Without going to a completely controlled environment like with callgrind, it would already be doable to have a less chaotic environment than travis virtual machines... On a "regular machine", timing test are easier to exploit. Another way to paliate for uncontrolled environments are stats, like we do in integration, or more elaborate ones like scientits do when they want to measure stuff that are even more chaotic than timings, e.g. humans ;)

[ayberkozgur]

There is no way of doing anything 100% properly ;)

No, the non-timing tests are completely 100% repeatable given the same inputs as long as there is no solar flare flipping your bits :)

[ayberkozgur]

8ee0e10a74626a6b6e8af23ed340f619b38fcdcc partly addressed this.