NatLibFi / Annif

Annif is a multi-algorithm automated subject indexing tool for libraries, archives and museums.
https://annif.org
Other
190 stars 41 forks source link

Increase maximum startup time threshold (0.3 -> 0.4 s) #725

Closed osma closed 11 months ago

osma commented 11 months ago

We have a CI test for startup time, implemented in PR #706, designed to prevent situations where basic Annif CLI commands suddenly take a long time due to e.g. slow imports.

The threshold value was set to 0.300 seconds. But nowadays this test quite often fails because the startup takes a bit longer, e.g. 0.35 seconds. I don't think anything has changed with the imports, it's just that the CI environment runs a bit slower than it used to.

This PR increases the threshold value to 0.400 seconds, which should prevent spurious test failures but still ensure that startup times have not increased dramatically.

sonarcloud[bot] commented 11 months ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

osma commented 11 months ago

Ironically, the CI startup time test failed because startup took .432 seconds.

codecov[bot] commented 11 months ago

Codecov Report

Patch and project coverage have no change.

Comparison is base (e9031bb) 99.67% compared to head (2709b55) 99.67%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #725 +/- ## ======================================= Coverage 99.67% 99.67% ======================================= Files 89 89 Lines 6397 6397 ======================================= Hits 6376 6376 Misses 21 21 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

osma commented 11 months ago

...and then a re-run succeeded, when the startup time had decreased to .297 seconds. The test is clearly not very robust, but I think it's better to have a slightly higher threshold to reduce the amount of spurious test failures, so I'm merging this.