Failure in JUnit mode with abstract JUnit 3.8 TestCase

TestNG Version

6.9.13 and later, including 7.3.0 (6.9.10 works, .11 and .12 are not available on Maven for some reason, so I did not test them)

Actual behavior

When the test suite contains both an abstract JUnit 3.8 TestCase with no public test methods AND a concrete JUnit 4 or TestNG test class with test methods, a "Failure in JUnit mode" is reported, but the build does not fail.

(The codebase I'm working in uses TestNG mostly, but there are a few old test classes remaining that are still based on a custom subclass of GroovyTestCase which in turn extends TestCase.)

Workarounds: rename the abstract class so Surefire doesn't detect it as a test class, or add an empty public test method to it.

Running TestSuite
Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@3abfe836
Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@42d8062c
[TestRunner] [ERROR] 
Failure in JUnit mode for class com.example.AbstractJUnit38TestCase
org.testng.TestNGException: 
Failure in JUnit mode for class com.example.AbstractJUnit38TestCase
    at org.testng.junit.JUnit4TestRunner.start(JUnit4TestRunner.java:114)
    at org.testng.junit.JUnit4TestRunner.run(JUnit4TestRunner.java:65)
    at org.testng.TestRunner$1.run(TestRunner.java:650)
    at java.util.ArrayList.forEach(ArrayList.java:1259)
    at org.testng.TestRunner.runJUnitWorkers(TestRunner.java:841)
    at org.testng.TestRunner.privateRunJUnit(TestRunner.java:679)
    at org.testng.TestRunner.run(TestRunner.java:583)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:384)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:378)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:337)
    at org.testng.SuiteRunner.run(SuiteRunner.java:286)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1218)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
    at org.testng.TestNG.runSuites(TestNG.java:1069)
    at org.testng.TestNG.run(TestNG.java:1037)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:77)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:175)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
    at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:106)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.NullPointerException
    at org.testng.internal.BaseTestMethod.<init>(BaseTestMethod.java:84)
    at org.testng.junit.JUnitTestMethod.<init>(JUnitTestMethod.java:17)
    at org.testng.junit.JUnit4TestMethod.<init>(JUnit4TestMethod.java:11)
    at org.testng.junit.JUnit4TestRunner.createTestResult(JUnit4TestRunner.java:242)
    at org.testng.junit.JUnit4TestRunner.access$100(JUnit4TestRunner.java:23)
    at org.testng.junit.JUnit4TestRunner$1.shouldRun(JUnit4TestRunner.java:91)
    at org.junit.internal.runners.JUnit38ClassRunner.filter(JUnit38ClassRunner.java:155)
    at org.junit.runner.manipulation.Filter.apply(Filter.java:97)
    at org.junit.internal.requests.FilterRequest.getRunner(FilterRequest.java:37)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
    at org.testng.junit.JUnit4TestRunner.start(JUnit4TestRunner.java:80)
    ... 29 more
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.274 sec

Note that the failure did not actually cause the build to fail (might be a duplicate of #311).

Test case sample

pom.xml

<project>
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>testng-abstract-junit38-sample</artifactId>
    <version>0.0.1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.3.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

src/test/java/com/example/AbstractJUnit38TestCase.java

package com.example;

import junit.framework.TestCase;

public abstract class AbstractJUnit38TestCase extends TestCase { }

src/test/java/com/example/ExampleTestNGTest.java

package com.example;

import org.testng.annotations.Test;

public class ExampleTestNGTest {

    @Test
    public void testNG() { }

}

testng-team / testng