thin-edge / thin-edge.io

The open edge framework for lightweight IoT devices
https://thin-edge.io
Apache License 2.0
221 stars 54 forks source link

flaky system test: Test if all c8y services are up in service_monitoring.robot #2427

Closed albinsuresh closed 9 months ago

albinsuresh commented 11 months ago

Describe the bug

Test failure in CI run: https://github.com/thin-edge/thin-edge.io/actions/runs/6783385820/job/18437730578?pr=2424

2023-11-07 11:13:28.840047 [PID:58115] [3] [ID:26] Execution failed in Tests.Cumulocity.Service Monitoring.Service Monitoring with 1 failing test(s)
==============================================================================
Tests                                                                         
==============================================================================
Tests.Cumulocity                                                              
==============================================================================
Tests.Cumulocity.Service Monitoring                                           
==============================================================================
Tests.Cumulocity.Service Monitoring.Service Monitoring                        
==============================================================================
Test if all c8y services are up                                       | FAIL |
AssertionError
------------------------------------------------------------------------------
Test if all c8y services are down                                     | PASS |
------------------------------------------------------------------------------
Test if all c8y services are using configured service type            | PASS |
------------------------------------------------------------------------------
Test if all c8y services using default service type when service t... | PASS |
------------------------------------------------------------------------------
Check health status of tedge-mapper-c8y service on broker stop start  | PASS |
------------------------------------------------------------------------------
Check health status of tedge-mapper-c8y service on broker restart ... | PASS |
------------------------------------------------------------------------------
Check health status of child device service :: Test service status... | PASS |
------------------------------------------------------------------------------
Tests.Cumulocity.Service Monitoring.Service Monitoring                | FAIL |
7 tests, 6 passed, 1 failed
==============================================================================
Tests.Cumulocity.Service Monitoring                                   | FAIL |
7 tests, 6 passed, 1 failed
==============================================================================

Screenshots

Robot Results

:white_check_mark: Passed :x: Failed :next_track_button: Skipped Total Pass % :stopwatch: Duration
355 1 3 356 99.72 57m29.898999999s

Failed Tests

Name Message :stopwatch: Duration Suite
Test if all c8y services are up AssertionError 174.204 s Service Monitoring

Environment (please complete the following information):

Additional context

reubenmiller commented 11 months ago

I suspect that this test suffers from the same problem as this other test in the same suite:

reubenmiller commented 9 months ago

Closing as issue has not been seen for 2 weeks after merging the fix. Please reopen if it is seen again