Error: Exception: dartfmt on lib/ failed with exit code -15 Exceeded timeout of 0:05:00.000000

[jonasfj]

Happens in analyzer. Probably we should report this as an error on the package results. Or maybe we should do something to limit how much code we analyze? Not sure...

Stack trace looks like:

pana/appengine/0.10.3: `dartfmt` failed.
Exception: dartfmt on lib/ failed with exit code -15
Exceeded timeout of 0:05:00.000000

Error:
    Exception: dartfmt on lib/ failed with exit code -15
    Exceeded timeout of 0:05:00.000000

Stack:
    #0      ToolEnvironment.filesNeedingFormat (package:pana/src/sdk_env.dart:231:7)
    <asynchronous suspension>
    #1      PackageAnalyzer._inspect (package:pana/src/package_analyzer.dart:161:26)
    <asynchronous suspension>
    #2      PackageAnalyzer.inspectPackage.<anonymous closure> (package:pana/src/package_analyzer.dart:92:22)
    <asynchronous suspension>
    #3      _rootRun (dart:async/zone.dart:1126:13)
    #4      _CustomZone.run (dart:async/zone.dart:1023:19)
    #5      _runZoned (dart:async/zone.dart:1518:10)
    #6      runZoned (dart:async/zone.dart:1465:12)
    #7      withLogger (package:pana/src/logging.dart:13:70)
    #8      PackageAnalyzer.inspectPackage (package:pana/src/package_analyzer.dart:78:12)
    #9      AnalyzerJobProcessor.process.analyze (package:pub_dev/analyzer/pana_runner.dart:75:31)
    <asynchronous suspension>
    #10     AnalyzerJobProcessor.process (package:pub_dev/analyzer/pana_runner.dart:96:36)
    <asynchronous suspension>
    #11     JobProcessor.run (package:pub_dev/job/job.dart:55:28)
    <asynchronous suspension>
    #12     JobMaintenance.run (package:pub_dev/job/job.dart:95:18)
    #13     _workerMain.<anonymous closure> (package:pub_dev/service/entrypoint/analyzer.dart:94:26)
    #14     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
    #15     StackZoneSpecification._registerUnaryCallback.<anonymous closure>.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:129:26)
    #16     StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
    #17     StackZoneSpecification._registerUnaryCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:129:14)
    #18     _rootRunUnary (dart:async/zone.dart:1134:38)
    #19     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
    #20     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
    #21     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
    #22     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
    #23     Future._completeWithValue (dart:async/future_impl.dart:524:5)
    #24     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:32:15)
    #25     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:290:13)
    #26     PopularityStorage.init (package:pub_dev/shared/popularity_storage.dart)
    #27     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
    #28     StackZoneSpecification._registerUnaryCallback.<anonymous closure>.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:129:26)
    #29     StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
    #30     StackZoneSpecification._registerUnaryCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:129:14)
    #31     _rootRunUnary (dart:async/zone.dart:1134:38)
    #32     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
    #33     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
    #34     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
    #35     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
    #36     Future._completeWithValue (dart:async/future_impl.dart:524:5)
    #37     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:32:15)
    #38     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:290:13)
    #39     PopularityStorage.fetch (package:pub_dev/shared/popularity_storage.dart)
    #40     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
    #41     StackZoneSpecification._registerUnaryCallback.<anonymous closure>.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:129:26)
...

[isoos]

Probably we should report this as an error on the package results.

We do report it, when it happens, it also gets a score penalty.

Or maybe we should do something to limit how much code we analyze?

It is an issue of packages with a lot of files (e.g. when generating from API definitions): dartdoc, dartfmt, dartanalyzer and in effect also pana's internal checks take much longer time. At the same time, as long as we allow the upload, I think we should attempt to analyze the packages without limiting it to a subset of it.

[Dokotela]

So does this mean that if I want to earn these points back on pub.dev I need to break my package into smaller pieces? I can run dartdoc and dartfmt on my local machine just fine, although it does take a while. When I run pana, I get the following

INFO       Running `/home/grey/dev/flutter/bin/cache/dart-sdk/bin/dartanalyzer --options /home/grey/.local/share/Trash/files/fhir.5/pana_analysis_options_1600959830439730.g.yaml --format machine lib`...
INFO       Running `/home/grey/dev/flutter/bin/cache/dart-sdk/bin/dartfmt --dry-run --set-exit-if-changed /home/grey/.local/share/Trash/files/fhir.5/lib`...
SEVERE     Killing `/home/grey/dev/flutter/bin/cache/dart-sdk/bin/dartfmt --dry-run --set-exit-if-changed /home/grey/.local/share/Trash/files/fhir.5/lib` Exceeded timeout of 0:05:00.000000
INFO       killed `/home/grey/dev/flutter/bin/cache/dart-sdk/bin/dartfmt --dry-run --set-exit-if-changed /home/grey/.local/share/Trash/files/fhir.5/lib` - true
SEVERE     `dartfmt` failed.
           Exception: dartfmt on lib/ failed with exit code -15

           Exception: dartfmt on lib/ failed with exit code -15

           #0      ToolEnvironment.filesNeedingFormat (package:pana/src/sdk_env.dart:246:9)
           <asynchronous suspension>
           #1      _formatPackage (package:pana/src/create_report.dart:220:52)
           #2      _staticAnalysis (package:pana/src/create_report.dart:102:15)
           <asynchronous suspension>
           #3      createReport (package:pana/src/create_report.dart:54:11)
           <asynchronous suspension>
           #4      PackageAnalyzer._inspect (package:pana/src/package_analyzer.dart:178:21)
           <asynchronous suspension>
           #5      PackageAnalyzer.inspectDir (package:pana/src/package_analyzer.dart:78:12)
           #6      main (file:///home/grey/.pub-cache/hosted/pub.dartlang.org/pana-0.14.3/bin/pana.dart:163:34)
           <asynchronous suspension>
           #7      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
           #8      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
INFO       Running `/home/grey/dev/flutter/bin/cache/dart-sdk/bin/pub get`...
INFO       Running `/home/grey/dev/flutter/bin/cache/dart-sdk/bin/pub outdated --json --up-to-date --no-dev-dependencies --no-dependency-overrides`...

The package I'm working with does contain a lot of files.

[isoos]

I can run dartdoc and dartfmt on my local machine just fine, although it does take a while.

Can you give use some numbers on how much time it does take?

[Dokotela]

For dartdoc:

real    9m3.876s
user    10m44.561s
sys     0m23.764s

For dartfmt:

real    18m1.047s
user    18m37.268s
sys     0m13.370s

[isoos]

/cc @jonasfj: wdyt, will we be able to support that much with sandboxing? I think we could also consider to increase the time limit only for latest-stable versions.

[jonasfj]

We could consider adjusting these...

Or we could investigate if there is a way to build the package to avoid these issues? maybe splitting it, maybe doing something else? Breaking it, into smaller packages is an option, maybe reducing the public interface, if there is things there that don't need to be public -- without suggesting that it's a poorly written package, maybe there is a way to fix this by tweaking the package. The case where I sometimes run into this is https://pub.dev/packages/googleapis which is huge, auto-generated, and well, maybe it should have been split.

@Dokotela what package are you working on? I assume it's published on pub.dev :)

[Dokotela]

Limiting this to stable versions makes some sense (and mine isn't, yet, so in that case it wouldn't apply).

I could split up the package, but there are a lot of times when someone may need the entire thing.

For me it is also certainly because of the amount of generated code. It is indeed published here: https://pub.dev/packages/fhir. It's for working with medical data using an international interoperability standard defined here: https://www.hl7.org/fhir/overview.html

Regardless of the decision, I appreciate the consideration.

[isoos]

Limiting this to stable versions makes some sense (and mine isn't, yet, so in that case it wouldn't apply).

From the pub site's point of view, you are releasing stable versions (as opposed to prerelease versions like 0.0.1-dev.0.1).

Increasing the time limit for latest stable version may work, but only if we find a way to schedule the smaller packages with higher priorities. One obvious measure would be the archive size, but it doesn't have the ratio about the .dart files vs other files like images or other assets.

[jonasfj]

There are many things that could be improved here:

• we could use faster machines,
• we could allow long execution time, and,
• analysis / dartdoc could be faster.

I think we'll gradually see improvements on all these fronts over time...

---

But since this currently affects a very small number of packages, and it looks like fhir has natural boundaries by which it could be split into smaller packages, splitting into smaller packages might make sense for now.

Another option is to generate the API docs and put it on github pages, and then point to it using the documentation: <url> in pubspec.yaml, see https://dart.dev/tools/pub/pubspec#documentation Doing so will at-least put a documentation link on pub.dev, it won't fix the score -- and given the low number of packages affected I doubt we'll prioritize this anytime, hopefully we'll eventually see faster machines, faster analysis and longer runtimes as we continue to improve our tooling and infrastructure.

If you do this, make sure to put the version number in the URL, and retain documentation for older versions of your package -- at-least that's a nice thing to do :D

[isoos]

Closing as no longer relevant - we have implemented new worker, new scheduling and new timeout handling since this issue was raised.