Inconsistent behaviour when doctests are absent

[fuelen]

Some time ago, module A had doctests, but now it doesn't have them and doctest A instruction is still in tests. When I run tests for the whole file, then everything is green. If I rerun test only for doctest, then it becomes red. Output is the following:

Excluding tags: [:test]
Including tags: [line: "4"]

All tests have been excluded.

Finished in 0.03 seconds (0.03s async, 0.00s sync)
23 tests, 0 failures, 23 excluded

Randomized with seed 487001

If I rerun tests for the whole file, then everything is green again.

You may ask why I rerun tests for doctests :) If another test in that file fails, and I run tests for the whole file, then doctest becomes red as well! I try to rerun doctests and consistently see it as red in neovim.

Expected behaviour: consider doctest instructions without tests as successful.

[jfpedroza]

There is a bit of a disconnection in the adapter between the part that detects the tests (in Lua), and the part that emits the results for tests (in Elixir).

When you run the whole file, Neotest is not expecting a result for the doctest and if all tests pass, the detected doctest passes too.

When you run the tests for the doctest, Neotest expects a result and if it doesn't arrive, it will mark it as failed. The Elixir part of the adapter, the ExUnit formatter, doesn't know about the doctest because ExUnit doesn't report anything about it as there are no tests in it.

It probably won't be simple to fix.