flutter / devtools

Performance tools for Flutter
https://flutter.dev/docs/development/tools/devtools/
BSD 3-Clause "New" or "Revised" License
1.54k stars 314 forks source link

Catch `devtools_server` regressions from flutter/devtools #8022

Open elliette opened 3 weeks ago

elliette commented 3 weeks ago

As a small improvement, having these tests run as part of the pre-submit instead of post-submit would be helpful.

However, it would be even more helpful if we could catch regressions on commits to flutter/devtools, so that these regressions are caught when they happen instead of last-minute as part of the release process, requiring cherrypicks to fix (e.g. https://github.com/dart-lang/sdk/issues/56128).

Brainstorming about how this could work

Another option is for us to update the DevTools version in the SDK more frequently than during our releases, so that these regressions will be caught sooner.

elliette commented 2 weeks ago

Would it be possible for LUCI to also change the SDK's DEPs to use that latest version of DevTools, and run the devtools_server tests against it?

@athomas - similar issue to github.com/flutter/devtools/issues/8023, do you know if this might be possible?

FYI @kenzieschmoll

athomas commented 1 week ago

Possible yes, but that's something for the new team. It makes the roller a bit more complex because it will need an SDK checkout. There may also be an out-of-the box solution with the Skia autoroller, but I'm not very familiar with that stack.