`waitForAnimationToEnd` isn't reliable

[bartekpacia]

Describe the bug

I have an app that displays a circular progress indicator for 31 seconds, and I expect waitForAnimationToEnd to spend its default timeout (15 seconds) waiting and only then exiting. But sometimes*, it returns too early, while the animation is still in progress.

*10 tries

OK = `waitForAnimationToEnd` returned after waiting for its default timeout (15s) ``` 1 OK 2 OK 3 NOT OK (✅ too early - after 5s) 4 NOT OK (✅ too early - after 5s) 5 NOT OK (✅ too early - after 8s) 6 NOT OK (✅ too early - after 8s) 7 NOT OK (✅ too early - after 9s) 8 OK 9 OK 10 NOT OK (✅ too early - after 7s) ```

Reproduce

flutter create a Flutter app:

flutter create flutter_sample --platforms android,ios

and paste the following code to main.dart:

Flutter app

```dart import 'package:flutter/material.dart'; void main() { runApp(const MyApp()); } class MyApp extends StatelessWidget { const MyApp({super.key}); @override Widget build(BuildContext context) { return MaterialApp( title: 'Flutter Demo', theme: ThemeData( colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple), useMaterial3: true, ), home: const MyHomePage(title: 'Flutter Demo Home Page'), ); } } class MyHomePage extends StatefulWidget { const MyHomePage({super.key, required this.title}); final String title; @override State createState() => _MyHomePageState(); } class _MyHomePageState extends State { int _counter = 0; final _future = Future.delayed(const Duration(seconds: 31)); void _incrementCounter() { setState(() { _counter++; }); } @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar( backgroundColor: Theme.of(context).colorScheme.inversePrimary, title: Text(widget.title), ), body: FutureBuilder( future: _future, builder: (context, snapshot) { if (snapshot.connectionState == ConnectionState.waiting) { return const Center( child: CircularProgressIndicator(), ); } return Center( child: Column( mainAxisAlignment: MainAxisAlignment.center, children: [ TextButton( onPressed: _incrementCounter, child: const Text('Increment'), ), const Text( 'You have pushed the button this many times:', ), Text( '$_counter', style: Theme.of(context).textTheme.headlineMedium, ), ], ), ); }, ), ); } } ``` and `flutter run`.

Then create flows/test.yaml:

Flow

```yaml appId: com.example.flutter_sample --- - clearState - launchApp - waitForAnimationToEnd - tapOn: Increment ``` and ` maestro test flows/test.yaml -c`.

Expected behavior

waitForAnimationToEnd always exits after 15 seconds (its default timeout), because the animation lasts 31 seconds (more than its default timeout).

Environment information (please complete the following information):

• Maestro version v.1.33.0
• Android API 31 emulator (Pixel 6)
• Flutter v3.13.4
• MacBook M1, macOS 13 Ventura

More context

waitForAnimationToEnd works by taking screenshots of the screen and comparing them. I suspect that when the circular progress indicator in the latter screenshot is in the similar position as in the former screenshot, Maestro considers the animation to have finished.

Waiting for animations to finish in Flutter

Flutter's testing framework has the pumpAndSettle() method, which will refresh the screen while there are frames scheduled (i.e. there's an animation in progress) and will return only once there are no more frames (i.e. the animation has finished).

Its downside is that it can result in false positives if the app is badly written (example). For example, displaying an animation that's invisible to the user still causes frames to be scheduled by the framework, and will result in pumpAndSettle() not returning even though nothing happens on the screen (from the user's perspecrive).

Perhaps a solution based on the above could fix the problem with waitForAnimationToEnd, at least in a Flutter app.

[amanjeetsingh150]

Flutter's testing framework has the pumpAndSettle() method, which will refresh the screen while there are frames scheduled (i.e. there's an animation in progress) and will return only once there are no more frames (i.e. the animation has finished).

Agreed the ideal situation would be to have this implemented in a flutter driver

[joshuadeguzman]

Hey @bartekpacia, I noticed this too. waitForAnimationToEnd does exit early. What were the solutions that you have tried that worked?

What's interesting is that it works locally well. But, it's flaky in CI, it's challenging to replicate the scenario with the simulators on the cloud where it "ends" early.

What I've done so far is:

• Forcing a wait, but still returns early for the animations
• Adding timeouts on the assertVisible (next screen validation)

I'll try finding an alternative for when I wait for the API response next.

[bartekpacia]

Hey @joshuadeguzman, i'm afraid there are no good solutions to it, apart from what you tried.

Agreed the ideal situation would be to have this implemented in a flutter driver

I think this is infeasible since we want Maestro to be fully framework and OS-independent, i.e. we want to only depend on accessibility tree.

Also implementing a flutter driver would be quite complex (see this article to learn more).