collect_coverage errors when isolates are spawned by the Dart script

[donny-dont]

I have a Dart library that does code generation that I am unable to reliably gather coverage from. The library has tests that rely on the output of an isolate spawned during the testing process.

The tests work as follows

1. Run code generation step
2. Spawn isolate which executes tests on the generated code which verifies they are working correctly.
3. If isolate returns without error the code generation worked, if there's an error then the code generation failed.

It appears that when running dart --observe=8000 tests/all.dart that the isolate being spawned by the test is waiting for a connection to the observe port before exiting.

Running pub global run coverage:collect_coverage --port=8000 -o coverage.json then causes the test to complete but then the following error occurs

JsonRpcError: VM must be paused (101)
dart:async                                       _Completer.completeError
package:coverage/src/devtools.dart 227           _Connection._handleResponse
===== asynchronous gap ===========================
dart:io                                          _WebSocketImpl.listen
package:coverage/src/devtools.dart 184           _Connection._Connection
package:coverage/src/devtools.dart 191           _Connection.connect.<async>
===== asynchronous gap ===========================
dart:async                                       _asyncThenWrapperHelper
package:coverage/src/devtools.dart               _Connection.connect
package:coverage/src/devtools.dart 93            VMService.connectToVMWebsocket.<async>
===== asynchronous gap ===========================
dart:async                                       Future.Future.microtask
package:coverage/src/devtools.dart               VMService.connectToVMWebsocket
package:coverage/src/devtools.dart 89            VMService.connect.<async>
===== asynchronous gap ===========================
dart:async                                       Future.Future.microtask
package:coverage/src/devtools.dart               VMService.connect
package:coverage/src/collect.dart 17             collect.<async>.<fn>
package:coverage/src/util.dart 38                retry.<async>.<fn>.<async>
===== asynchronous gap ===========================
package:coverage                                 retry.<async>.<fn>
package:coverage/src/util.dart 17                retry.<async>._withTimeout
package:coverage/src/util.dart 35                retry.<async>
===== asynchronous gap ===========================
dart:async                                       Future.Future.microtask
package:coverage/src/util.dart                   retry
package:coverage/src/collect.dart 16             collect.<async>
===== asynchronous gap ===========================
dart:async                                       Future.Future.microtask
package:coverage/src/collect.dart                collect
http://localhost:50032/collect_coverage.dart 21  main.<async>.<fn>.<async>
===== asynchronous gap ===========================
package:stack_trace                              Chain.capture
http://localhost:50032/collect_coverage.dart 20  main.<async>

After running the command multiple times the coverage.json appears to be generated.

I'm not sure if this is a problem due to how dart is invoked or if coverage should be doing something different.

My setup is

sdk 1.14.0-dev.6.0
coverage 0.7.2
Windows 7

The repository I stumbled upon the issue is https://github.com/dogma-dart/dogma-codegen/.

A smaller use case is https://github.com/dogma-dart/dogma-codegen-test which is used by the above project to do this workflow. It has a set of tests as well that will not work with the coverage library.

I did have success with this environment for other projects so I think this just revolves around the isolate handling.

@kevmoo

[kevmoo]

@donny-dont could you look to see if this still reproduces?

[donny-dont]

I can give it a go locally. I actually moved away from doing things this way because I migrated to build and because the bug was sitting here.