NatLibFi / Annif

Annif is a multi-algorithm automated subject indexing tool for libraries, archives and museums.
https://annif.org
Other
190 stars 41 forks source link

Test CLI startup time with CI/CD job #706

Closed juhoinkinen closed 1 year ago

juhoinkinen commented 1 year ago

Adds a GitHub Actions job with a shell script for measuring the CLI startup time to prevent its unnoticed deterioration.

The startup time was optimized in #696, and now it is good make sure the startup remains fast. For example, I noticed that adding type hints (planned in #690) can slow the startup, but this can be avoided by postponing the evaluation of annotations.

The timing script executes annif --help four times and measures user + sys time. If the average of the user + sys time is over a threshold, the job fails. The threshold is set to 0.300 s.

In the five previous runs the measured time was 0.245 s, 0.240 s, 0.252 s, 0.242 s and 0.245 s, but in some earlier runs the measured time was over 0.300 s. So failures can be expected, however I think it does not matter if they are reasonably rare (say ~10% of runs).

Failure of this job does not make the other jobs fail, so unit testing, linting and publishing are run in any case.

sonarcloud[bot] commented 1 year ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

codecov[bot] commented 1 year ago

Codecov Report

Patch and project coverage have no change.

Comparison is base (6f13121) 99.66% compared to head (881c270) 99.66%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #706 +/- ## ======================================= Coverage 99.66% 99.66% ======================================= Files 89 89 Lines 6295 6295 ======================================= Hits 6274 6274 Misses 21 21 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.