[NETBEANS-6] Adding Support for JUnit 5

mcdonnell-john commented 6 years ago

Main Features include:

Added JUnit 5 Library - Using latest 5.3.1 release.
Added JUnit 5 Templates
New Test classes default to JUnit 5 if JDK >= 8 & JUnit 4 not already used.
Test Classes when JUnit 5 is found on the class path(Maven) use JUnit 5.
Run/Debug single test method support included. Checks when Using JUnit 5 of the method has an annotation that inherits from @Testable
Updated JUnit IDE hints for JUnit 5 - i.e. Assertions as opposed to Assert and ordering of arguements.
Informational message to users when creating a Test Suite with Junit 5 - Currently not supported with Jupiter, message refers users to switch to the Vintage Engine.

sormuras commented 6 years ago

Nice to see JUnit 5 support in NB, soon! ;-)

Using Testable as the signal to show users "testable" elements is good.

Just scrolled through the PR and didn't find a single "junit-platform-launcher" snippet. How does NetBeans execute tests? Via the command line? Or Maven/Surefire?

mcdonnell-john commented 6 years ago

Hi @sormuras,

So I toy-ed around for a while looking to see if I could implement an EngineDiscoveryRequest but couldn't get it to work, and then went back to the @Testable check based on its JavaDocs and left it at that for now.

When using Maven based projects, Netbeans uses Maven/Surefire - I added a quick check for Surefire >2.22.0 for supporting individual test methods. I don't use Ant for any project, so would need help from someone who does to see what's needed to support changes for Ant.

sormuras commented 6 years ago

I see. So, at the moment, users see something like "Run test..." in the IDE/editor and when click, a Maven run tailored to the action is executed. Correct? If yes, it's good starting point.

You could use the junit-platform-console launcher, like I do here: https://github.com/sormuras/junit-platform-maven-plugin/tree/master/src/main/java/de/sormuras/junit/platform/maven/plugin -- then you are less dependent on how Surefire finds and executes tests.

Regarding Ant and its configuration in NB ... methinks, it would need to include junit-platform-console and use the provided launcher.

Or...

You use the Launcher API like described here: https://junit.org/junit5/docs/current/user-guide/#launcher-api

How can I test-drive this PR? Checkout your "apache/incubator-netbeans" clone, switch to branch feature/JUnit5 and "Press Play on Tape"?

sormuras commented 6 years ago

Answering my last question on my own: http://netbeans.apache.org/download/index.html#source

mcdonnell-john commented 6 years ago

Thanks for the comments.

NetBeans is closely tied to maven, and for JUnit surefire - I tried to keep as close to the existing solution that exists for JUnit 4 as I could to keep the changes as simple as possible.

I think the biggest issue with this is the fact that surefire doesn't support the @DisplayName annotation, since the reporting in NetBeans also uses the output from Surefire from what I have seen. - So that's a gap we have with the maven support for JUnit 5 in NetBeans

Yes, "How can I test-drive this PR? Checkout your "apache/incubator-netbeans" clone, switch to branch feature/JUnit5 and"... - run "ant tryme"

sormuras commented 6 years ago

Regarding @DisplayName and Surefire: working on it already https://issues.apache.org/jira/projects/SUREFIRE/issues/SUREFIRE-1546

sormuras commented 6 years ago

FYI: there is an issue with Surefire keeps using version 5.2 of JUnit (JUnit Platform 1.2.0) although you override the version in a pom.xml file. See https://issues.apache.org/jira/projects/SUREFIRE/issues/SUREFIRE-1564 for details.

mcdonnell-john commented 6 years ago

Regarding @DisplayName and Surefire: working on it already https://issues.apache.org/jira/projects/SUREFIRE/issues/SUREFIRE-1546

Cool, When this comes in, then we may have to look at the Test Results window in NetBeans to show the names explicitly.

sormuras commented 6 years ago

If stars align, it should work out-of-the-box. Just update to the yet to be released Surefire version and the ReportEntry object will report the display names instead of the "legacy reporting names", i.e. the name of the test methods and classes.

geertjanw commented 6 years ago

Great, if I can do something to try this out, would be great -- maybe you could update this page with the list of features provided for JUnit 5 here: https://cwiki.apache.org/confluence/display/NETBEANS/Miscellaneous

geertjanw commented 6 years ago

Any reason not to merge this, what still needs to be done, etc?

mcdonnell-john commented 6 years ago

Any reason not to merge this, what still needs to be done, etc?

Hi, I don't think there is.

geertjanw commented 6 years ago

Can you add screenshots to illustrate these features to that Miscellaneous page?

mcdonnell-john commented 6 years ago

Will do - should be able to get to that on Sunday...

Saljack commented 6 years ago

Hi, why JUnit BOM is not used in pom.xml for versions and dependencies? It looks better and more clear in my oppinion?

  </dependencies>
    <!--JUNIT 5-->
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-params</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>5.3.1</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

mcdonnell-john commented 6 years ago

Hi, why JUnit BOM is not used in pom.xml for versions and dependencies? It looks better and more clear in my oppinion?

Because the current NetBeans implementation of adding libraries to an existing pom doesn't support that, as far as I could see. I tried to limit my changes to re-use a lot of what already worked with JUnit 4.

It's also a matter of taste, and to me its not a big deal.

geertjanw commented 6 years ago

Great, and thanks for the screenshots (https://cwiki.apache.org/confluence/display/NETBEANS/Miscellaneous). What is left to be done, for this release, and for subsequent releases?

mcdonnell-john commented 6 years ago

HI @geertjanw I cant think of anything else, I mostly focused on Maven support and maybe some ant stuff was handle there, but there probably some work there if someone who uses ant wants to take on.

I think for this release we should be okay, and we'll need to update the Test Specs for JUnit testing for NetCat to help test this a little more.

Moving forward, for subsequent releases: What ever is missing for Ant support, Better test suite generation - when Jupiter supports it

Apart from that I don't see anything as Surefire should be fixing their display name annotation handling issue, if that doesn't need any additional changes then great, if not we can include that in a future release.

geertjanw commented 6 years ago

Great. Let's merge if no one objects 24 hours from now.

Tibor17 commented 6 years ago

We want to cut a new Surefire Version 2.22.1 for JUnit5 5.2.0+. This will support 5.3.0 too of course and Java 11. We have decided that version 3.0.0-M1 will adopt Display Name. We won't to postpone the release 2.22.1 too much.

geertjanw commented 5 years ago

I took a look at JUnit 5 and Ant in NetBeans, some thoughts here:

https://blogs.apache.org/netbeans/entry/junit-5-apache-ant-and

sormuras commented 5 years ago

Probably the best solution for Ant users in Apache NetBeans would be to be able to continue to use JUnit 4, rather than JUnit 5, since the latter is a total nightmare to set up in the context of Ant, as described above. No need to endure nightmares...

Have you tried the all-in-one-fat-jar solution with Ant? Download junit-platform-console-standalone and that's all.

Copied from the User Guide

junit-platform-console-standalone An executable JAR with all dependencies included is provided at Maven Central under the junit-platform-console-standalone directory. See Console Launcher for details.

geertjanw commented 5 years ago

Ah, OK, that's great.

Could we bundle that with Apache NetBeans, in the same way that we're currently bundling jupiter-api, jupiter-params, and jupiter-engine? And could we remove those three and replace it with the fat JAR? Is it licensed in the same way?

sormuras commented 5 years ago

Could we bundle that with Apache NetBeans,...

Perhaps. It's a "all-in" solution. Could Maven projects still refer to the modules and use them while Ant projects get the junit-platform-console-standalone support?

Is it licensed in the same way?

It states:

Open Source Licenses
====================

This product may include a number of subcomponents with separate
copyright notices and license terms. Your use of the source code for
these subcomponents is subject to the terms and conditions of the
subcomponent's license, as noted in the LICENSE-<subcomponent>.md
files.

It contains the following licences:

Eclipse Public License - v 1.0 (junit 3 + 4)
Eclipse Public License - v 2.0 (junit 5)
BSD License (hamcrest)
Apache License 2.0 (picocli, univocity-parsers)

sormuras commented 5 years ago

@geertjanw Here is an example of a "patched" Ant installation that allows Ant projects to be configured hassle-free:

https://github.com/junit-team/junit5-samples/tree/master/junit5-jupiter-starter-ant

See https://github.com/junit-team/junit5-samples/blob/master/junit5-jupiter-starter-ant/build.sh#L6-L20 for how to "install" the junit-platform-console-standalone.jar into Ant (as a lib) and here is how you may use the built-in task and the console launcher side-by-side: https://github.com/junit-team/junit5-samples/blob/master/junit5-jupiter-starter-ant/build.xml#L35-L61

geertjanw commented 5 years ago

Yes, I've been studying this one a lot, it's great and very useful:

https://github.com/junit-team/junit5-samples/blob/master/junit5-jupiter-starter-ant/build.xml

And the org.junit.platform.console.ConsoleLauncher works fine, produces results, etc. The only question is how to hook it into the Results Window in NetBeans, which is done very well for Surefire, which is why JUnit 5 with Maven in NetBeans works so smoothly in this regard.

sormuras commented 5 years ago

[...] which is done very well for Surefire [...]

By a listener that Surefire calls when tests are executed?

Nevertheless and as you have written in your blog, NetBeans should use the JUnit Platform Launcher API to start test runs. That's the sole purpose of it -- no need to depend on a plugin of a specific build tool that tries to "find and execute" tests in a "custom" way. Copied from the beginning of the User Guide:

The JUnit Platform serves as a foundation for launching testing frameworks on the JVM. It also defines the TestEngine API for developing a testing framework that runs on the platform. Furthermore, the platform provides a Console Launcher to launch the platform from the command line and build plugins for Gradle and Maven as well as a JUnit 4 based Runner for running any TestEngine on the platform.

https://junit.org/junit5/docs/current/user-guide/index.html#overview-what-is-junit-5

And every new TestEngine, wrapping an existing test framework (mainrunner, TestNG, Spock, JQWIK, ... https://github.com/junit-team/junit5/wiki/Third-party-Extensions#junit-platform-test-engines) or one that is invented tomorrow (JUnit 6 "aka Saturn") will be integrated in your build tool right-away. Zero effort.

apache / netbeans

[NETBEANS-6] Adding Support for JUnit 5 #892