flutter / devtools

Performance tools for Flutter
https://flutter.dev/docs/development/tools/devtools/
BSD 3-Clause "New" or "Revised" License
1.58k stars 326 forks source link

Speed up integration testing on the CI (taking around ~21 right now 09/23) #5938

Open kenzieschmoll opened 1 year ago

kenzieschmoll commented 1 year ago

example run that took 47 min: https://github.com/flutter/devtools/actions/runs/5336567626/jobs/9671473410?pr=5919

We should see if there is anything we can do to shard these or share steps that we are already doing for other jobs (e.g. code generation). Perhaps we could run the test in 3-4 different shards and make each shard its own job that can run concurrently. We may also consider sharing the step to start chrome driver and shut it down, as right now this is happening for each test, not once for running all tests. This is not that expensive though looking at the timing logs, but it could be a small win.

kenzieschmoll commented 1 year ago

After the above two referenced PRs, the longest integration test job now takes 27 min.

kenzieschmoll commented 1 year ago

In this example run before sharding, the integration tests had the following runtimes:

After https://github.com/flutter/devtools/pull/6385, the run times for the flutter and flutter-web devices (where we've added sharding) are down significantly:

kenzieschmoll commented 1 year ago

We should continue to investigate how to drive this time down further by sharing more prep tasks across jobs (getting deps or generating mocks).