Coverage not reported correctly for too many line hits

[Mr-Pepe]

I was told to report this issue here.

Steps to reproduce

• Run dart create dart_test

• Add the following to lib/dart_test.dart

  String doSomething(bool flag) {
  if (flag == true) {
    return 'true';
  }
  
  if (flag == false) {
    return 'false';
  }
  
  throw ArgumentError('Invalid flag');
  }

• Add the following to test/dart_test_test.dart (sorry for the name...)

  import 'package:dart_test/dart_test.dart';
  import 'package:test/test.dart';
  
  void main() {
  test('Test', () {
    const nTests = 100000;
    for (var i = 0; i < nTests; i++) {
      expect(doSomething(true), equals('true'));
    }
    expect(doSomething(false), equals('false'));
  });
  }

• Execute tests with dart run coverage:test_with_coverage && genhtml coverage/lcov.info -o coverage/html

Expected behavior

The line if (flag == false) should always be counted as covered, regardless of nTests.

Observed behavior

For nTests = 10, the line is counted as covered.

❯ dart run coverage:test_with_coverage && genhtml coverage/lcov.info -o coverage/html
The Dart VM service is listening on http://127.0.0.1:8181/
The Dart DevTools debugger and profiler is available at: http://127.0.0.1:8181/devtools?uri=ws://127.0.0.1:8181/ws
00:00 +1: All tests passed!                                                                                                                                                                            
Found 1 entries.
Found common filename prefix "/home/felipe/tmp/dart_test"
Generating output.
Processing file lib/dart_test.dart
  lines=4 hit=3
Overall coverage rate:
  lines......: 75.0% (3 of 4 lines)
  functions......: no data found

For nTests = 100000, the line is counted as not covered.

❯ dart run coverage:test_with_coverage && genhtml coverage/lcov.info -o coverage/html
The Dart VM service is listening on http://127.0.0.1:8181/
The Dart DevTools debugger and profiler is available at: http://127.0.0.1:8181/devtools?uri=ws://127.0.0.1:8181/ws
00:00 +1: All tests passed!                                                                                                                                                                            
Found 1 entries.
Found common filename prefix "/home/felipe/tmp/dart_test"
Generating output.
Processing file lib/dart_test.dart
  lines=4 hit=2
Overall coverage rate:
  lines......: 50.0% (2 of 4 lines)
  functions......: no data found

Placing the second expect expect(doSomething(false), equals('false')); before the loop leads to the line always being counted as covered.

System

[✓] Flutter (Channel stable, 3.19.2, on Arch Linux 6.7.6-arch1-2, locale en_US.UTF-8)
    • Flutter version 3.19.2 on channel stable at /home/felipe/snap/flutter/common/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 7482962148 (5 days ago), 2024-02-27 16:51:22 -0500
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /home/felipe/Android/Sdk/
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /home/felipe/Applications/android-studio/jbr/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • CHROME_EXECUTABLE = chromium

[✓] Linux toolchain - develop for Linux desktop
    • clang version 10.0.0-4ubuntu1
    • cmake version 3.16.3
    • ninja version 1.10.0
    • pkg-config version 0.29.1

[✓] Android Studio (version 2022.2)
    • Android Studio at /home/felipe/Applications/android-studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] Connected device (2 available)
    • Linux (desktop) • linux  • linux-x64      • Arch Linux 6.7.6-arch1-2
    • Chrome (web)    • chrome • web-javascript • Chromium 122.0.6261.94 Arch Linux

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[Mr-Pepe]

I have investigated this a bit and it seems that the coverage report provided by the VM service is already faulty. I guess that makes it a dart-lang/sdk issue.