Make sleep() time tests more robust

gul-cpp / gul14

General Utility Library for C++14

https://gul14.info/

GNU Lesser General Public License v2.1

2 stars 1 forks source link

Make sleep() time tests more robust #14

Closed alt-graph closed 2 years ago

alt-graph commented 2 years ago

This MR increases the tolerance for time measurements after sleep() in unit tests. It also improves the documentation of several sleep() overloads to make it clear that the functions introduce a delay that is at least as long as specified.

Many sleep() related tests fail on GitHub MacOS runners because the systems are under heavy contention. As sleep() is an interruption point, context switches sometimes lead to delays on the order of several tens of milliseconds. We adapt to that by increasing the tolerance after sleep() from 18 to 100 ms.

soerengrunewald commented 2 years ago

How sure are we that 100ms are sufficient and we not have to change it soon again?

Finii commented 2 years ago

I would

Just include tests that check what we guarantee (is later than)
Make other tests (not later than) optional

In this way we would test always only what we really guarantee. But can enable the additional tests on platforms where we are confident that they are fast enough.

alt-graph commented 2 years ago

I am not a big fan of optional tests. Strictly speaking, we should only test what we guarantee. There may always be an occasion in which a stupid context switch takes a second. I'll remove the "not-later-than" tests.

alt-graph commented 2 years ago

I force-pushed a new version of the PR. There is only one test left with a "no-later-than" condition, and that is needed to check that we can interrupt sleep()from another thread.