Pub run test fails in CI

[AKushWarrior]

https://travis-ci.org/github/bcgit/pc-dart/builds/697600045

Something about command pub run test is failing. Will dig in to see what's up later.

[mwcw]

I had a bit of a dig into it last night.

My machine: (Dart VM version: 2.8.4 (stable) (Wed Jun 3 12:26:04 2020 +0200) on "macos_x64") I got the same failure once after running it several times.

On our internal CI we run: (Dart VM version: 2.8.2 (stable) (Unknown timestamp) on "linux_x64")

dart --version pub get pub run test pub global activate pana export PATH="$PATH":"$HOME/.pub-cache/bin" pana --verbosity compact --source path .

And it failed once out of several reruns.

On Travis: Mostly failure but when it did run something set the return code to non-zero even though the test runner reported "All tests passed!"

[mwcw]

Further:

I manually triggered a build of master and the test runner did not fail.

https://travis-ci.org/github/bcgit/pc-dart/builds/697853171

[AKushWarrior]

Very strange. I wonder what could cause inconsistency in tests with no network code?

[mwcw]

Hi,

I am fairly certain that this is not us but here is the trace from my own development machine.

 pc-dart % pub --trace run test .
The method 'path' was called on null.
Receiver: null
Tried calling: path("bin/test.dart")
dart:core                                   Object.noSuchMethod
package:pub/src/executable.dart 117:60      _executablePath
package:pub/src/executable.dart 83:32       runExecutable.<fn>
package:pub/src/log.dart 377:32             warningsOnlyUnlessTerminal
package:pub/src/executable.dart 64:16       runExecutable
package:pub/src/command/run.dart 78:26      RunCommand.run
package:args/command_runner.dart 197:27     CommandRunner.runCommand
package:pub/src/command_runner.dart 191:39  PubCommandRunner.runCommand.<fn>
dart:async                                  new Future.sync
package:pub/src/utils.dart 113:12           captureErrors.wrappedCallback
package:stack_trace                         Chain.capture
package:pub/src/utils.dart 126:11           captureErrors
package:pub/src/command_runner.dart 191:13  PubCommandRunner.runCommand
===== asynchronous gap ===========================
dart:async                                  Future.catchError
package:pub/src/utils.dart 113:52           captureErrors.wrappedCallback
package:stack_trace                         Chain.capture
package:pub/src/utils.dart 126:11           captureErrors
package:pub/src/command_runner.dart 191:13  PubCommandRunner.runCommand
---- Log transcript ----
FINE: Pub 2.8.4
ERR : The method 'path' was called on null.
    | Receiver: null
    | Tried calling: path("bin/test.dart")
FINE: Exception type: NoSuchMethodError
ERR : dart:core                                   Object.noSuchMethod
    | package:pub/src/executable.dart 117:60      _executablePath
    | package:pub/src/executable.dart 83:32       runExecutable.<fn>
    | package:pub/src/log.dart 377:32             warningsOnlyUnlessTerminal
    | package:pub/src/executable.dart 64:16       runExecutable
    | package:pub/src/command/run.dart 78:26      RunCommand.run
    | package:args/command_runner.dart 197:27     CommandRunner.runCommand
    | package:pub/src/command_runner.dart 191:39  PubCommandRunner.runCommand.<fn>
    | dart:async                                  new Future.sync
    | package:pub/src/utils.dart 113:12           captureErrors.wrappedCallback
    | package:stack_trace                         Chain.capture
    | package:pub/src/utils.dart 126:11           captureErrors
    | package:pub/src/command_runner.dart 191:13  PubCommandRunner.runCommand
    | ===== asynchronous gap ===========================
    | dart:async                                  Future.catchError
    | package:pub/src/utils.dart 113:52           captureErrors.wrappedCallback
    | package:stack_trace                         Chain.capture
    | package:pub/src/utils.dart 126:11           captureErrors
    | package:pub/src/command_runner.dart 191:13  PubCommandRunner.runCommand
---- End log transcript ----

[AKushWarrior]

In the CI, you could call dart test/all_tests_web.dart. That should run the whole test suite.

[AKushWarrior]

I'm not at my machine so I can't confirm that this method works locally.

[mwcw]

Yes it works locally.

meganwoods@ pc-dart % dart test/all_tests_web.dart 
00:00 +0: StreamCipherAsBlockCipher: Null: cipher  : Lorem ipsum dolor sit amet[...]
00:00 +1: StreamCipherAsBlockCipher: Null: cipher  : En un lugar de La Mancha, [...]

etc.

[AKushWarrior]

Maybe try that in Travis CI? Back at my computer now, so I can run any tests you need on Mac/Linux.

As far as I can tell, all_tests_web runs all of the relevant tests.

[mwcw]

I'll set it up..

[mwcw]

Hi

This has not manifested again, I will close this for now.

MW