maiflai / gradle-scalatest

A plugin to enable the use of scalatest in a gradle Scala project.
The Unlicense
73 stars 35 forks source link

Only the "deepest" test names are printed during test execution #73

Closed netvl closed 4 years ago

netvl commented 5 years ago

If you use FreeSpec, you can define nested tests like this:

"Some thing" - {
  "when some condition holds" - {
    "should do whatever" in {
      <..test code..>
    }
  }
}

Here the in block is the actual test which is run, while the enclosing statements provide context. When you run the tests e.g. with SBT, test execution output looks like this:

Some thing
  when some condition holds
    - should do whatever

With Gradle scalatest plugin, however, only the actual test block name is printed:

- should do whatever

I couldn't find any way to enable the full output approach with configuration. Is this a bug or intentional behavior?

maiflai commented 5 years ago

Hi,

This sounds like a bug - please could you try running the command line with --console plain?

Thanks, Stu

netvl commented 5 years ago

I observe exactly the same behavior with --console plain, although test block names appear to be indented correctly. Still no context messages, though.

netvl commented 5 years ago

Could this be related to test reports generation? In the project where I have HTML reports generation disabled (in order not to add a dependency to pegdown), I observe this issue. In another project, which does have a pegdown test runtime dependency, I see the test descriptions printed out correctly.

maiflai commented 5 years ago

Ah - that is interesting, what is the issue preventing the use of pegdown on the other project please?

Would it be possible to add it temporarily?

netvl commented 5 years ago

Oh, sorry for misleading - it seems that I was wrong. I was under impression that that first project did have HTML reports disabled, but it seems this is no longer the case since some time ago (we have pegdown in the testRuntime scope now). I still observe the issue even then :(

maiflai commented 5 years ago

Please could you supply a sample project? I don't seem to be able to reproduce this on the command line of macOS with Gradle 5.2.1 and scalatest 3.0.5.

Task :scalatest Discovery starting. Discovery completed in 89 milliseconds. Run starting. Expected test count is: 1 FreeTest: Some thing when some condition holds - should do whatever (13 milliseconds) Run completed in 184 milliseconds. Total number of tests run: 1 Suites: completed 2, aborted 0 Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0 All tests passed.

netvl commented 4 years ago

A year has passed since - sorry for the long delay! It just happened that I worked on another project which did not exhibit this behavior, so it did not bother me much.

However, now I believe I know the reason. This behavior - whether to print context messages or not - is controlled by the TestLogEvent.STARTED option, whose absence maps to -oNHP arguments to the Scalatest runner. I see that the Scalatest plugin sets all TestLogEvent values, but apparently one of our internal plugins resets them back to a limited subset, which does not include TestLogEvent.STARTED. The thing is, in one of our projects we have our internal plugin applied before the Scalatest plugin, but in others it is the other way around. This naturally results in different enabled event configuration.

So I guess this could be closed :) thanks!

maiflai commented 4 years ago

Great, thanks very much.

Stu