search

dart-lang / test

A library for writing unit tests in Dart.
https://pub.dev/packages/test
491 stars 211 forks source link

Test suites showing as <no name> instead of filename in IntelliJ IDEA and Android Studio #2213

Open chris-NR opened 2 months ago

chris-NR commented 2 months ago

When running dart tests in IntelliJ and Android studio the 'test suite' is shown as '\<no name>' in the Test Runner results panel (see screenshot 1)

Steps to reproduce

  1. From the 'IDEA Welcome' dialog choose 'New Project'
  2. From the 'New Project' dialog choose 'Dart Package'
  3. Click 'Next' and provide a \<project name>
  4. Click 'Create' and the project opens
  5. In the 'Project' panel select the 'test' directory
  6. Right-Click and select 'Run tests in \<project name>'
  7. Choose 'Show passed' in the Test Runner panel
  8. Observe test suite '\<no name>' instead of 'project name_test.dart'

Notes

  1. This does not happen in vscode (see screenshot 2)
  2. This does not happen if you use 'flutter_test' instead (see screenshot 3)
  3. There is a stale bug logged against IDEA here
  4. Screenshot 2 (vscode) also shows a minor bug where the dart analyser has incorrectly flagged 'dependencies' in pubspec.yaml

Tool Versions (info also shown in screenshots)

Android Studio Iguana | 2023.2.1
Build #AI-232.10227.8.2321.11479570, built on February 21, 2024

IntelliJ IDEA 2024.1 (Community Edition)
Build #IC-241.14494.240, built on March 28, 2024

Screenshots

Screenshot 2024-04-24 at 14 02 02 Screenshot 2024-04-24 at 14 16 23 Screenshot 2024-04-24 at 14 41 17
jakemac53 commented 1 month ago

There is an implicit unnamed test group at the outer layer of every test suite - which seems to be getting visualized here.

My assumption would be that VsCode just knows about that, and is replacing it with the name of the test, cc @DanTup for confirmation.

We wouldn't want to give that a real name (such as the test path), because it would mess up the command line reporting when you run dart test (this already visualizes the test path separately from the name of the test).

DanTup commented 1 month ago

My assumption would be that VsCode just knows about that, and is replacing it with the name of the test, cc @DanTup for confirmation.

Not quite - the root node for the suite exists before we get any results, but we have code that squashes out unnamed groups at any level (we add them to a mapping with their parent - null for top level - and then whenever we look up a parent, we check that map first).

It would be nice if we could handle this better pkg:test because editors are less likely to have different behaviour then, but since changes to something like this could affect existing integrations, probably the time to do something like that would be bundled together with other breaking changes to the "API" the editors use. (I don't have any in mind, although I have long dreamed of the pkg:test API being more of a long-running service I can ask for lists of tests, or to run them... maybe DTD will make that kind of thing easier to avoid IDEs having yet more background processes to spawn).