thin-edge / thin-edge.io

The open edge framework for lightweight IoT devices
https://thin-edge.io
Apache License 2.0
221 stars 54 forks source link

Flaky test mapper_fails_during_sw_update_recovers_and_process_response test #1879

Closed albinsuresh closed 9 months ago

albinsuresh commented 1 year ago

Describe the bug

The test tedge_mapper::c8y::tests::mapper_fails_during_sw_update_recovers_and_process_response is failing quite frequently on cargo test runs.

To Reproduce

Run cargo test -p tedge-mapper c8y::tests::mapper_fails_during_sw_update_recovers_and_process_response repeatedly. It fails at least once in every three attempts.

Additional Context

It was reported in the past in #570 and fixed with https://github.com/thin-edge/thin-edge.io/pull/575

didier-wenzek commented 1 year ago
failures:

---- c8y::tests::mapper_fails_during_sw_update_recovers_and_process_response stdout ----
thread 'c8y::tests::mapper_fails_during_sw_update_recovers_and_process_response' panicked at 'Didn't receive all expected messages: ["503,c8y_SoftwareUpdate,\n"]
 Received: ["500\n", "105", "500\n", "105", "102,test-device_c8y-mapper-test-client,\"service\",c8y-mapper-test-client,\"down\"", "102,test-device_c8y-mapper-test,\"service\",c8y-mapper-test,\"down\"", "102,test-device_c8y-mapper-test-client,\"service\",c8y-mapper-test-client,\"down\"", "102,test-device_c8y-mapper-test-client,\"service\",c8y-mapper-test-client,\"up\""]', /home/runner/work/thin-edge.io/thin-edge.io/crates/tests/mqtt_tests/src/test_mqtt_client.rs:89:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
didier-wenzek commented 1 year ago

Another test is flaky: cargo test -p tedge_file_system_ext

failures:

 ---- tests::test_fs_events stdout ----
thread 'tests::test_fs_events' panicked at 'assertion failed: `(left == right)`
  left: `Some(Modified("/tmp/.tmpuU7RpW/dir_b/file_b"))`,
 right: `Some(Modified("/tmp/.tmpuU7RpW/file_a"))`', /home/runner/work/thin-edge.io/thin-edge.io/crates/core/tedge_actors/src/test_helpers.rs:274:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:

---- tests::test_fs_events stdout ----
thread 'tests::test_fs_events' panicked at 'assertion failed: `(left == right)`
  left: `None`,
 right: `Some(Modified("/tmp/.tmpvLggGZ/file_a"))`', /home/runner/work/thin-edge.io/thin-edge.io/crates/core/tedge_actors/src/test_helpers.rs:274:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
didier-wenzek commented 1 year ago

Yet another one - from the system test suite

Failed Tests

Name Message Duration Suite
Software list should be populated during startup Managed object does not have a c8y_SoftwareList fragment 165.316 s Software
didier-wenzek commented 1 year ago

This PR temporary ignores the flaky integration tests: https://github.com/thin-edge/thin-edge.io/pull/1882.

gligorisaev commented 1 year ago

QA has thoroughly checked the feature and here are the results:

didier-wenzek commented 9 months ago

Closing as these tests have been definitely removed.