search

flutter / flutter

Flutter makes it easy and fast to build beautiful apps for mobile and beyond
https://flutter.dev
BSD 3-Clause "New" or "Revised" License
166.66k stars 27.6k forks source link

Mac tool_integration_tests_5_5 is 2.00% flaky #156962

Open fluttergithubbot opened 1 month ago

fluttergithubbot commented 1 month ago

The post-submit test builder Mac tool_integration_tests_5_5 had a flaky ratio 2.00% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1049 Commit: https://github.com/flutter/flutter/commit/4f2211b550daef2d6073bb36542040f1f7156d51

Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1049 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1022

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

Please follow https://github.com/flutter/flutter/blob/master/docs/infra/Reducing-Test-Flakiness.md#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

fluttergithubbot commented 3 weeks ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-10-23 and 2024-10-29 is 2.00%. Flaky number: 2; total number: 100. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1239 Commit: https://github.com/flutter/flutter/commit/5879a5cba13741a170c18a91f9583f6483c8dd60 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1239 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1200

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

fluttergithubbot commented 2 weeks ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-10-28 and 2024-11-05 is 1.00%. Flaky number: 1; total number: 100. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1325,%201324 Commit: https://github.com/flutter/flutter/commit/9a5e8ce6b52c8e8a8864270c45ea6a82497e7bb2 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1325 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1324

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

bkonyi commented 2 weeks ago

We're seeing similar failures on this PR: https://github.com/flutter/flutter/pull/157100

bkonyi commented 2 weeks ago

This looks related to https://github.com/dart-lang/native/issues/1698.

fluttergithubbot commented 1 week ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-04 and 2024-11-12 is 1.06%. Flaky number: 1; total number: 94. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1390 Commit: https://github.com/flutter/flutter/commit/2e23f7b30ea79180a3507c53058a21c2d856e4e5 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1390

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

Piinks commented 5 days ago

(Triage): @bkonyi are we still tracking this? Thanks!

fluttergithubbot commented 4 days ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-12 and 2024-11-19 is 2.06%. Flaky number: 2; total number: 97. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1557 Commit: https://github.com/flutter/flutter/commit/966aeb28ea04d5166ad40a85ca20f7b093d70260 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1557 https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_5_5/1520

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_5_5

bkonyi commented 4 days ago

I thought this was resolved as of https://github.com/flutter/flutter/pull/158946, but apparently we're still missing something... :(