search

troglobit / sysklogd

BSD syslog daemon with syslog()/syslogp() API replacement for Linux, RFC3164 + RFC5424
https://troglobit.com/sysklogd.html
Other
93 stars 20 forks source link

Tests are flaky on low-power devices and CPU-limited containers #63

Open enter-github-username opened 1 year ago

enter-github-username commented 1 year ago

Some tests, like test/facility.sh, use hard-coded sleep statements to ensure that prior operations finish.

While running the test suite of sysklogd inside a heavily CPU-limited podman container, a few tests (test/memleak.sh test/facility.sh test/secure.sh test/remote.sh) showed flaky behaivior, i.e. they failed occasionally. The same behaivior showed on an old notebook with a > 10 year old i3 CPU running Debian 11. This may be a minor problem for CI environments, which often heavily restrict the available CPU time.

A simple solution would be to just sleep longer - however, this would increase the test suite runtime significantly, and I would assume that only a very small amount of users are affected by the flakyness in the first place. It may be a better idea to just add it to the documentation as a caveat.

Alternatively, it may be a good idea to, in case of an unexpected failure like an empty grep result, recheck and sleep a few times inside the tests and only fail after n retries.

troglobit commented 1 year ago

Yes, they are flaky. I have not had the time to fix that, use them mainly for regression testing when modifying the code base. What are you using them for?