trivago / cucable-plugin

Maven plugin that simplifies running Cucumber scenarios in parallel.
Apache License 2.0
115 stars 42 forks source link

Problem in running multiple tags from maven command line #124

Closed Naumansh closed 5 years ago

Naumansh commented 5 years ago

mvn clean verify -Ddevice=chrome -Dtags=@validLogin2,@validLogin3

I have been using the above way to run tests using tags. Same is suggested by you in an older issue https://github.com/trivago/cucable-plugin/issues/41.

However when multiple tags are added, tests are not running without any error. Below is the snipped of config in pom.


    <properties>
        <tags/>
</properties>

<plugin>
                <groupId>com.trivago.rta</groupId>
                <artifactId>cucable-plugin</artifactId>
                <version>${cucable.version}</version>
                <executions>
                    <execution>
                        <id>generate-test-resources</id>
                        <phase>generate-test-resources</phase>
                        <goals>
                            <goal>parallel</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <sourceRunnerTemplateFile>src/test/java/sba/sbir/steps/CustomCreator.java</sourceRunnerTemplateFile>
                    <sourceFeatures>src/test/resources/features</sourceFeatures>
                    <generatedFeatureDirectory>${generated.feature.directory}</generatedFeatureDirectory>
                    <generatedRunnerDirectory>${generated.runner.directory}</generatedRunnerDirectory>
                    <includeScenarioTags>${tags}</includeScenarioTags>
                    <parallelizationMode>scenarios</parallelizationMode>
                    <logLevel>off</logLevel>
                </configuration>
            </plugin> ```

PS: It runs perfectly fine when a single tag is provided.
bischoffdev commented 5 years ago

This is not how multiple tags need to be added. The correct approach from version 1.5.0 on is

-Dtags="@validLogin2 and @validLogin3"

If you want to run either the first or the second, you can do

-Dtags="@validLogin2 or @validLogin3"

because it uses Cucumber expressions.

Naumansh commented 5 years ago

Thanks man.

1- It is still not running with "-Dtags="@validLogin2 and @validLogin3" with above shared configuration. i.e <includeScenarioTags>${tags}</includeScenarioTags>

2- However if the above tag is removed. The scenarios starts to execute but below ugle logs are generated first. Console Logs: [INFO] Running TestSuite [TestNG] [ERROR] No test suite found. Nothing to run Usage:

[options] The XML suite files to run Options: -configfailurepolicy Configuration failure policy (skip or continue) -d Output directory -dataproviderthreadcount Number of threads to use when running data providers -excludegroups Comma-separated list of group names to exclude -groups Comma-separated list of group names to be run -junit JUnit mode Default: false -listener List of .class files or list of class names implementing ITestListener or ISuiteListener -methods Comma separated of test methods Default: [] -methodselectors List of .class files or list of class names implementing IMethodSelector -mixed Mixed mode - autodetect the type of current test and run it with appropriate runner Default: false -objectfactory List of .class files or list of class names implementing ITestRunnerFactory -parallel Parallel mode (methods, tests or classes) Possible Values: [tests, methods, classes, instances, none, true, false] -port The port -reporter Extended configuration for custom report listener -suitename Default name of test suite, if not specified in suite definition file or source code -suitethreadpoolsize Size of the thread pool to use to run suites Default: 1 -testclass The list of test classes -testjar A jar file containing the tests -testname Default name of test, if not specified in suitedefinition file or source code -testnames The list of test names to run -testrunfactory, -testRunFactory The factory used to create tests -threadcount Number of threads to use when running tests in parallel -usedefaultlisteners Whether to use the default listeners Default: true -log, -verbose Level of verbosity -xmlpathinjar The full path to the xml file inside the jar file (only valid if -testjar was specified) Default: testng.xml

bischoffdev commented 5 years ago

Hi! You probably should ignore your TestSuite class in the failsafe options when invoking your test run.

Naumansh commented 5 years ago

I tried below but still getting same logs


                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>${maven.failsafe.plugin.version}</version>
                <executions>
                    <execution>
                        <id>Run parallel tests</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <threadCount>${threads}</threadCount>
                    <testFailureIgnore>true</testFailureIgnore>
                    <disableXmlReport>true</disableXmlReport>
                    <argLine>-Dfile.encoding=UTF-8</argLine>
                    <excludes>
                        <exclude>**/IT*.java</exclude>
                        <exclude>**/testng.xml</exclude>
                    </excludes>
                </configuration>
            </plugin>```
Naumansh commented 5 years ago

I know its not related to cucable but i would really appreciate your suggested solution on it.

bischoffdev commented 5 years ago

Closing this as it is not related to Cucable.

I'm sorry but I am not an expert in testng. Maybe there's something here? https://stackoverflow.com/questions/50132094/no-test-suite-found-nothing-to-run-but-testng-xml-running-properly-from-eclip