TESTING: `make qtest` fails due to `pause=0`

[Lutra-Fs]

If I run manually it will be fine but failed while running make qtest test suite

Console Log

rattleng on  lutra/195-dataset-re-engineer-displaydart-to-clarify-the-logic [$] is 📦 v6.3.0+123 via 🎯 v3.5.0 took 1m19s
❯ make qtest
========================================
integration_test/app_test.dart
integration_test/app_test.dart========================================
00:00 +0: loading E:/rattleng/integration_test/app_test.dart                                                                                                                                                                          B00:28 +0: loading E:/rattleng/integration_test/app_test.dart                                                                                                                                                                     26.2s
√ Built build\windows\x64\runner\Debug\rattle.exe
00:31 +0: App Startup.
R START                  main.R
00:32 +1: App Startup.
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following StateError was thrown running a test (but after the test had completed):
Bad state: Tried to read a provider from a ProviderContainer that was already disposed

When the exception was thrown, this was the stack:
#0      ProviderContainer.readProviderElement (package:riverpod/src/framework/container.dart:423:7)
#1      ProviderElementProxy.read (package:riverpod/src/framework/proxy_provider_listenable.dart:129:26)
#2      ProviderContainer.read (package:riverpod/src/framework/container.dart:241:21)
#3      ProviderElementBase.read (package:riverpod/src/framework/element.dart:675:23)
#4      ptyProvider.<anonymous closure>.<anonymous closure> (package:rattle/providers/pty.dart:54:9)
#15     _StringAdapterSink.add (dart:convert/string_conversion.dart:228:11)
#16     _StringAdapterSink.addSlice (dart:convert/string_conversion.dart:233:7)
#17     _Utf8ConversionSink.addSlice (dart:convert/string_conversion.dart:307:20)
#18     _Utf8ConversionSink.add (dart:convert/string_conversion.dart:300:5)
#19     _ConverterStreamEventSink.add (dart:convert/chunked_conversion.dart:69:18)
#27     CastStreamSubscription._onData (dart:_internal/async_cast.dart:85:11)
#39     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)
(elided 28 frames from dart:async and package:stack_trace)
════════════════════════════════════════════════════════════════════════════════════════════════════
00:32 +0 -1: App Startup. [E]
  Test failed. See exception logs above.
  The test description was: App Startup.

  This test failed after it had already completed.
  Make sure to use a matching library which informs the test runner
  of pending async work.

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following StateError was thrown running a test (but after the test had completed):
Bad state: Tried to read a provider from a ProviderContainer that was already disposed

When the exception was thrown, this was the stack:
#0      ProviderContainer.readProviderElement (package:riverpod/src/framework/container.dart:423:7)
#1      ProviderElementProxy.read (package:riverpod/src/framework/proxy_provider_listenable.dart:129:26)
#2      ProviderContainer.read (package:riverpod/src/framework/container.dart:241:21)
#3      ProviderElementBase.read (package:riverpod/src/framework/element.dart:675:23)
#4      ptyProvider.<anonymous closure>.<anonymous closure> (package:rattle/providers/pty.dart:54:9)
#15     _StringAdapterSink.add (dart:convert/string_conversion.dart:228:11)
#16     _StringAdapterSink.addSlice (dart:convert/string_conversion.dart:233:7)
#17     _Utf8ConversionSink.addSlice (dart:convert/string_conversion.dart:307:20)
#18     _Utf8ConversionSink.add (dart:convert/string_conversion.dart:300:5)
#19     _ConverterStreamEventSink.add (dart:convert/chunked_conversion.dart:69:18)
#27     CastStreamSubscription._onData (dart:_internal/async_cast.dart:85:11)
#39     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)
(elided 28 frames from dart:async and package:stack_trace)
════════════════════════════════════════════════════════════════════════════════════════════════════
  Test failed. See exception logs above.
  The test description was: App Startup.

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following StateError was thrown running a test (but after the test had completed):
Bad state: Tried to read a provider from a ProviderContainer that was already disposed

When the exception was thrown, this was the stack:
#0      ProviderContainer.readProviderElement (package:riverpod/src/framework/container.dart:423:7)
#1      ProviderElementProxy.read (package:riverpod/src/framework/proxy_provider_listenable.dart:129:26)
#2      ProviderContainer.read (package:riverpod/src/framework/container.dart:241:21)
#3      ProviderElementBase.read (package:riverpod/src/framework/element.dart:675:23)
#4      ptyProvider.<anonymous closure>.<anonymous closure> (package:rattle/providers/pty.dart:54:9)
#15     _StringAdapterSink.add (dart:convert/string_conversion.dart:228:11)
#16     _StringAdapterSink.addSlice (dart:convert/string_conversion.dart:233:7)
#17     _Utf8ConversionSink.addSlice (dart:convert/string_conversion.dart:307:20)
#18     _Utf8ConversionSink.add (dart:convert/string_conversion.dart:300:5)
#19     _ConverterStreamEventSink.add (dart:convert/chunked_conversion.dart:69:18)
#27     CastStreamSubscription._onData (dart:_internal/async_cast.dart:85:11)
#39     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)
(elided 28 frames from dart:async and package:stack_trace)
════════════════════════════════════════════════════════════════════════════════════════════════════
  Test failed. See exception logs above.
  The test description was: App Startup.

00:32 +0 -1: Some tests failed.
 NO *****
========================================
integration_test/dataset_demo_test.dart
integration_test/dataset_demo_test.dart========================================
00:00 +0: loading E:/rattleng/integration_test/dataset_demo_test.dart                                                                                                                                                                 BNo tests ran.
|Terminate batch job (Y/N)? 
^Cmake: *** [support/flutter.mk:278: qtest] Error 512
PS>^C

rattleng on  lutra/195-dataset-re-engineer-displaydart-to-clarify-the-logic [$] is 📦 v6.3.0+123 via 🎯 v3.5.0 took 45s
❯ flutter test .\integration_test\app_test.dart
Connected devices:
Windows (desktop) • windows • windows-x64    • Microsoft Windows [Version 10.0.26120.1542]
Edge (web)        • edge    • web-javascript • Microsoft Edge 129.0.2792.52
[1]: Windows (windows)
[2]: Edge (edge)
Please choose one (or "q" to quit): 1
00:00 +0: loading E:/rattleng/integration_test/app_test.dart                                                                                                                                                                          B00:25 +0: loading E:/rattleng/integration_test/app_test.dart                                                                                                                                                                     23.6s
√ Built build\windows\x64\runner\Debug\rattle.exe
00:28 +0: App Startup.
R START                  main.R
00:29 +1: All tests passed!

Environment

rattleng on  lutra/195-dataset-re-engineer-displaydart-to-clarify-the-logic [$] is 📦 v6.3.0+123 via 🎯 v3.5.0 
❯ $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.5
PSEdition                      Core
GitCommitId                    7.4.5
OS                             Microsoft Windows 10.0.26120
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[Lutra-Fs]

It appears the error is occurring due to the --dart-define=PAUSE=0 argument. @gjwgit, do you have any insights on this?

[gjwgit]

We are seeing this at times - the R script is perhaps not finishing before we move on. If make dataset_demo.qtest or make dataset_demo.itest succeed then it is perhaps a timing issue and can consider the test to be passed for now or better, add appropriate delays where required.

[Lutra-Fs]

The whole integration test suite is too slow. Already took 32 min in my machine and not yet finished a run of make qtest.

[gjwgit]

It is slow . Need to investigate if it can be sped up.

I have renames pause to interact in dev to emphasise that this delay is an optional delay meant for manually watching a test. If a delay really is required then use delay (or hack if we think it should not require a delay here and so marked to get fixed).

[gjwgit]

CLosing for now unless there is a specific task to undertake here.