TestBench depends on commercial components

[probert94]

I just started using Vaadin TestBench to create UI Unit-Tests (possible since Vaadin 23.2). Unfortunately my tests all fail, because I don't have Vaadin Charts included in my project, as I don't need them at the moment. The problem is, that the base class UIUnitTest implements the interface TesterWrappers which itself imports Charts and some othe components.
My current workaround is to import com.vaadin:vaadin for scope test and use com.vaadin:vaadin for other scopes.

[mvysny]

Thank you for letting us know! This bug is very easy to reproduce:

1. Take the https://github.com/vaadin/skeleton-starter-flow project
2. Add a dependency on vaadin-testbench-unit-junit5 as described at https://vaadin.com/docs/latest/testing/ui-unit/getting-started
3. Add the following Java unit test:

   
   package org.vaadin.example;

import com.vaadin.testbench.unit.UIUnitTest; import org.junit.jupiter.api.Test;

public class MainViewTest extends UIUnitTest {

@Test
public void setText_clickButton_notificationIsShown() {
}

}

Upon running this unit-test, the runner fails with:

Internal Error occurred. org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-jupiter' failed to discover tests at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:132) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:107) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:78) at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:110) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86) at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86) at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53) at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57) at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55) Caused by: org.junit.platform.commons.JUnitException: ClassSelector [className = 'org.vaadin.example.MainViewTest'] resolution failed at org.junit.platform.launcher.listeners.discovery.AbortOnFailureLauncherDiscoveryListener.selectorProcessed(AbortOnFailureLauncherDiscoveryListener.java:39) at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:103) at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.run(EngineDiscoveryRequestResolution.java:83) at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolver.resolve(EngineDiscoveryRequestResolver.java:113) at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:46) at org.junit.jupiter.engine.JupiterTestEngine.discover(JupiterTestEngine.java:69) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152) ... 13 more Caused by: java.lang.NoClassDefFoundError: com/vaadin/flow/component/charts/Chart at java.base/java.lang.Class.getDeclaredMethods0(Native Method) at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3402) at java.base/java.lang.Class.privateGetPublicMethods(Class.java:3427) at java.base/java.lang.Class.getMethods(Class.java:2019) at org.junit.platform.commons.util.ReflectionUtils.getDefaultMethods(ReflectionUtils.java:1521) at org.junit.platform.commons.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:1494) at org.junit.platform.commons.util.ReflectionUtils.findMethod(ReflectionUtils.java:1343) at org.junit.platform.commons.util.ReflectionUtils.isMethodPresent(ReflectionUtils.java:1244) at org.junit.jupiter.engine.discovery.predicates.IsTestClassWithTests.hasTestOrTestFactoryOrTestTemplateMethods(IsTestClassWithTests.java:50) at org.junit.jupiter.engine.discovery.predicates.IsTestClassWithTests.test(IsTestClassWithTests.java:46) at org.junit.jupiter.engine.discovery.ClassSelectorResolver.resolve(ClassSelectorResolver.java:67) at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.lambda$resolve$2(EngineDiscoveryRequestResolution.java:135) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1602) at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129) at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:647) at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:189) at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:126) at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:92) ... 18 more Caused by: java.lang.ClassNotFoundException: com.vaadin.flow.component.charts.Chart at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ... 42 more

[mvysny]

The problem is exactly as @probert94 describes: the function ChartTester<Chart> test(Chart chart) in TesterWrappers.java causes the TesterWrappers class to fail to load properly.

Unfortunately, the only solution I see is to remove the offending function from TesterWrappers class (or to move it somewhere else), which is a breaking change (but only if the project is using Charts, which limits the scope of the breaking change).

My suggestion would be to move the function to a CommercialTesterWrapper class.

1. it would be a class and not an interface, since importing utility functions by implementing an interface is considered an anti-pattern;
2. the functions would be imported via a static import of CommercialTesterWrapper.*
3. The ChartTester class might have to move to a separate jar file, since BaseUIUnitTest.scanForTesters("com.vaadin.flow.component") might fail to load the ChartTester class, causing the class BaseUIUnitTest to fail to initialize; but this remains to be tested.
4. scanForTesters() could alternatively silently ignore testers for classes that are not present on the classpath; and maybe log a warning.

@caalador could you please provide your thoughts?

[caalador]

Basically the end result for the junit test would be to get the component parts to the flow-components repository to the components the same way that testbench-element is under the component. And then have the TesterWrappers in platform in the respective packages (vaadin/vaadin-core).

This has not been seen as an high issue as TestBench and unit testing is also commercial. @mcollovati did we have some plan for this or was it fully undermined by the TesterWrappers helper class?

[mvysny]

True - maybe the better way to tackle this would be to mandate to have com.vaadin:vaadin on the test classpath, since if you already have a license, you probably want to use it to its fullest.

[mcollovati]

If I recall correctly, one idea was to split the TesterWrappers in two, one for core components and the other one (extending the first) for the commercial components. But that would force the developer to add an implements CommercialTesterWrappers to their UI Unit test class.

[akashsharma3030]

When this bug will be fixed if you guys can provide update it would be great