goss-org / goss

Quick and Easy server testing/validation
https://goss.rocks
Apache License 2.0
5.53k stars 470 forks source link

Prometheus goss_tests_run_{outcomes, duration} metric has wrong label if the goss suite has no errors/skipped tests #789

Closed timeu closed 7 months ago

timeu commented 1 year ago

Describe the bug

The two metrics goss_tests_run_outcomes_total and goss_tests_run_duration_milliseconds for the overall run always have the "unknown" outcome label if the goss test has no failures or skipped tests. It works fine, if there is at least one skipped or failed test

# HELP goss_tests_run_duration_milliseconds The end-to-end duration of this run.
# TYPE goss_tests_run_duration_milliseconds counter
goss_tests_run_duration_milliseconds{outcome="unknown"} 126
# HELP goss_tests_run_outcomes_total The outcomes of this run as a whole.
# TYPE goss_tests_run_outcomes_total counter
goss_tests_run_outcomes_total{outcome="unknown"} 5

How To Reproduce

Expected Behavior

goss_tests_run_outcomes_total{outcome="pass"} 1 
goss_tests_run_duration_milliseconds{outcome="pass"} xx

Actual Behavior

goss_tests_run_outcomes_total{outcome="unknown"} 1 
goss_tests_run_duration_milliseconds{outcome="unknown"} zzz

Environment: goss 0.3.20

petemounce commented 1 year ago

Note to self; documentation content for manual.

aelsabbahy commented 1 year ago

Going to mark this as approved, since it seems like a confirmed ticket from @petemounce and @timeu

@petemounce out of curiosity, what's the level of effort to fix this?

petemounce commented 1 year ago

I think the effort would be low; this sounds like an omission to initialise the metrics being exported.

So, a unit (or integration might be easier) test to assert the expectation, and then a fix.