Closed mathias21 closed 8 years ago
It is solved here https://github.com/mathias21/cucumber-reporting
@mathias21 how did u solve it? should i clone and build from the link you provided
Hi @saikrishna321
I just downloaded the library Cucumber-Reporting that this one is using to generate the reports, I modified the code and the tests adding some extra code and I compiled it with maven.
You have to name your json reports like this: deviceName_somethingelse[otherstuff].json
You can download my code, go to the main folder and type mvn package (install maven first if you don't have it). It will automatically generate in your target folder the .jar file for the library (cucumber-reporting-0.x.x-SNAPSHOT.jar). Then you have to copy that file to your jenkins: /jenkins-data/plugins/cucumber-reports/WEB-INF/lib/ with the same name it was there before.
Good luck, any questions let me know ^_^
If you send me a pull request I will merge your solution into the next release
Sent from my iPhone
On 16 Jun 2014, at 11:31, Jorge notifications@github.com wrote:
Hi @saikrishna321
I just downloaded the library Cucumber-Reporting that this one is using to generate the reports, I modified the code and the tests adding some extra code and I compiled it with maven.
You have to name your json reports like this: deviceName_somethingelse[otherstuff].json
You can download my code, go to the main folder and type mvn package (install maven first if you don't have it). It will automatically generate in your target folder the .jar file for the library (cucumber-reporting-0.x.x-SNAPSHOT.jar). Then you have to copy that file to your jenkins: /jenkins-data/plugins/cucumber-reports/WEB-INF/lib/ with the same name it was there before.
Good luck, any questions let me know ^_^
— Reply to this email directly or view it on GitHub.
Hi @kingsleyh
It is not optimised and it is making assumptions with the input files. Let me improve it. So far I'm quite busy with automation and Jenkins. I will let you know.
Thank you!
@mathias21 i tried to build the way you suggested mvm package i get the below error
Sais-MacBook-Air-2:cucumber-reporting-master saikrishna$ mvn package
[INFO] Scanning for projects...
[INFO]
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building cucumber-reporting 0.0.24-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ cucumber-reporting ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ cucumber-reporting ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 10 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ cucumber-reporting ---
[INFO] Compiling 24 source files to /Users/saikrishna/Downloads/cucumber-reporting-master/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ cucumber-reporting ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 19 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ cucumber-reporting ---
[INFO] Compiling 13 source files to /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ cucumber-reporting ---
[INFO] Surefire report directory: /Users/saikrishna/Downloads/cucumber-reporting-master/target/surefire-reports
Running net.masterthought.cucumber.ArtifactProcessorTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.033 sec
Running net.masterthought.cucumber.DocStringTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.214 sec
Running net.masterthought.cucumber.ElementTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.153 sec
Running net.masterthought.cucumber.FeatureTest
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.267 sec
Running net.masterthought.cucumber.JsChartUtilTest
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec
Running net.masterthought.cucumber.ReportBuilderTest
FILENAME: masterthought-example-ATM.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATM.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATM.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
FILENAME: masterthought-example-ATMK.feature-project3.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/project3.json
com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Unterminated object at line 19 column 18
java.lang.NullPointerException
FILENAME: tictactoe.feature-docstring.json.html
JSON: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/docstring.json
Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.414 sec <<< FAILURE!
shouldRenderTheFeaturePageCorrectly(net.masterthought.cucumber.ReportBuilderTest) Time elapsed: 0.271 sec <<< ERROR!
java.io.FileNotFoundException: /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/masterthought-example-ATM.feature.html (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.
Running net.masterthought.cucumber.ReportInformationTest Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.487 sec Running net.masterthought.cucumber.ReportParserTest [WARNING] scenario has no steps: some name Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec Running net.masterthought.cucumber.RowTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec Running net.masterthought.cucumber.StatusesTest Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec Running net.masterthought.cucumber.StepTest [WARNING] Line 7 : Step is missing Result: Given : a "PSA" PSA customer [WARNING] Line 8 : Step is missing Result: When : PSA customer make the InitSession request to the Perseus [WARNING] Line 9 : Step is missing Result: Then : the Perseus must respond with the response message and return code 201 to PSA [WARNING] Line 10 : Step is missing Result: And : the initSessionRequest is set to "true" and TTS is set to "true" [WARNING] Line 11 : Step is missing Result: And : the tpeg response is set to "true" and the message format is "TPEGBinary" for PSA [WARNING] Line 12 : Step is missing Result: And : the expected App is "SAA", "WEA", "TEC" in PSA [WARNING] Line 13 : Step is missing Result: When : PSA customer make the GetMessage request to the Perseus [WARNING] Line 14 : Step is missing Result: Then : the Perseus must respond with the response message and return code 200 to PSA [WARNING] Line 15 : Step is missing Result: And : the get message content length is greater than 100 in PSA [WARNING] Line 7 : Step is missing Result: Given : a "PSA" PSA customer [WARNING] Line 8 : Step is missing Result: When : PSA customer make the InitSession request to the Perseus [WARNING] Line 9 : Step is missing Result: Then : the Perseus must respond with the response message and return code 201 to PSA [WARNING] Line 10 : Step is missing Result: And : the initSessionRequest is set to "true" and TTS is set to "true" [WARNING] Line 11 : Step is missing Result: And : the tpeg response is set to "true" and the message format is "TPEGBinary" for PSA [WARNING] Line 12 : Step is missing Result: And : the expected App is "SAA", "WEA", "TEC" in PSA [WARNING] Line 13 : Step is missing Result: When : PSA customer make the GetMessage request to the Perseus [WARNING] Line 14 : Step is missing Result: Then : the Perseus must respond with the response message and return code 200 to PSA [WARNING] Line 15 : Step is missing Result: And : the get message content length is greater than 100 in PSA [WARNING] Line 7 : Step is missing Result: Given : a "PSA" PSA customer [WARNING] Line 7 : Step is missing Result: Given : a "PSA" PSA customer [WARNING] Line 7 : Step is missing Result: Given : a "PSA" PSA customer Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.083 sec Running net.masterthought.cucumber.TagsTest Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.05 sec
Results :
Tests in error: shouldRenderTheFeaturePageCorrectly(net.masterthought.cucumber.ReportBuilderTest): /Users/saikrishna/Downloads/cucumber-reporting-master/target/test-classes/net/masterthought/cucumber/masterthought-example-ATM.feature.html (No such file or directory)
Tests run: 124, Failures: 0, Errors: 1, Skipped: 0
[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 7.075 s [INFO] Finished at: 2014-06-16T19:08:47+05:30 [INFO] Final Memory: 12M/81M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project cucumber-reporting: There are test failures. [ERROR] [ERROR] Please refer to /Users/saikrishna/Downloads/cucumber-reporting-master/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Hi @saikrishna321
My mistake, I'm using my own environment and I'm not taking care if everything works perfectly. I made mvn package before, and that generated all files for me, that is why I was not facing this problem with Tests.
Let me fix this. I will post the solution as soon as possible.
Hi @saikrishna321
Try now. It is passing Tests for me. Let me know if it is not working. Small mistake I made, I forgot to add the check in: /cucumber-reporting/src/main/java/net/masterthought/cucumber/json/Feature.java, line 52.
The assertions for the tests were failing because the name should be other. Now it should pass the tests.
Hi @kingsleyh
I modified this plugin and when I tried to compile it with Maven I got this error:
[ERROR] Failed to execute goal on project cucumber-reports: Could not resolve dependencies for project net.masterthought.jenkins:cucumber-reports:hpi:0.0.24-SNAPSHOT: Could not find artifact com.googlecode.totallylazy:totallylazy:jar:1077 in repo.jenkins-ci.org (http://repo.jenkins-ci.org/public/) -> [Help 1]
It looks like this link is not working anymore: http://repo.maven.apache.org/maven2/com/googlecode/totallylazy/totallylazy/1077/totallylazy-1077.jar
Can you help me with this? I would like to test some new features I added to allow parallel testing with Cucumber-reporting.
Thank you.
How are you getting your dependencies?
The totally lazy code is not on any maven repo except the bodar one so if you are going thru a corporate system they might not be allowing you out to bodar repo?
Otherwise just download the dependency locally - there's an issue about this on one of my cucumber plugin projects where someone posted a solution
Sorry I can't help more but at work at the moment
Sent from my iPhone
On 19 Jun 2014, at 15:42, Jorge notifications@github.com wrote:
Hi @kingsleyh
I modified this plugin and when I tried to compile it with Maven I got this error:
[ERROR] Failed to execute goal on project cucumber-reports: Could not resolve dependencies for project net.masterthought.jenkins:cucumber-reports:hpi:0.0.24-SNAPSHOT: Could not find artifact com.googlecode.totallylazy:totallylazy:jar:1077 in repo.jenkins-ci.org (http://repo.jenkins-ci.org/public/) -> [Help 1]
Can you help me with this? I would like to test some new features I added to allow parallel testing with Cucumber-reporting.
Thank you.
— Reply to this email directly or view it on GitHub.
Sorry I'm new with Maven but I understand a few things:
I didn't modified the POM file so I believe all dependencies are being taken from the defined places. Can you please write the link with the solution?
Thank you!
Hi @kingsleyh
I just modified the POM file, adding the following:
<repositories> <repository> <id>repo.bodar.com</id> <url>http://repo.bodar.com</url> </repository> </repositories>
<dependencies> <dependency> <groupId>com.googlecode.totallylazy</groupId> <artifactId>totallylazy</artifactId> <version>1077</version> </dependency> </dependencies>
Then I deleted the line number 35 from cucumber-reports.iml:
'<orderEntry type="library" name="Maven: com.googlecode.totallylazy:totallylazy:1077" level="project" />'
Using mvn clean package, it worked. Now I'm trying to test it using my Jenkins server.
Hi @kingsleyh
I made 2 pull requests to your projects (cucumber-reporting and this one). It would be great if you can merge both to allow everyone to use parallel testing with cucumber reports.
Thank you!
Hi
Thanks very much for your contributions. I'm snowed under with work at the moment do haven't had a change to do much yet.
I hope to find some time next week to merge and release.
--k
Sent from my iPhone
On 9 Jul 2014, at 16:18, Jorge notifications@github.com wrote:
Hi @kingsleyh
I made 2 pull requests to your projects (cucumber-reporting and this one). It would be great if you can merge both to allow everyone to use parallel testing with cucumber reports.
Thank you!
— Reply to this email directly or view it on GitHub.
@mathias21 now i get this work. I can see the counts with respect feature. But as i'm running parallel test on devices(calabash-mobile automation scripts) i don't get to know which results is for which device.
Can you throw some light on this.
Hi @saikrishna321
I think the problem now is your execution command. This is the one I'm using, you have to specify the output json file to make it work, always keeping the device name at the beginning, splitting report, time, and so on, with _.
ADB_DEVICE_ARG=#{device_id} TEST_SERVER_PORT=#{port} SCREENSHOT_PATH=#{screenshots_path} calabash-android run #{FILE.apk} --format json -o #{device_name}report#{time}.json
Where #{} are variables where you can drop your names/config.
Good luck!
@mathias21 Thanks a lot for the info. This is exactly the way i'm running and my json reports are named as device_id.json
Now when all the json reports are loading in report(for example: device_id_1.json,device_id_2.json,device_id_2.json). All these reports will have the same Step Definition as they are executed in parallel.
Now device_id_2.json has a failure on the same step definition and the report shows fail.How do i get to know that failure is from device_id_2 for that step.
Don't split device_idX with "". Use another char, the library is taking the first token from the json name splitting by _, so only device will appear inside Device Name. If you change this, I believe this should work. Can you post an image of what is happening?
^_^
Thanks @mathias21 i'm going to try this n will let you know. Currentyly i don't have access to the reports which has this issue. Will try to get this on local machine n paste the image.
@mathias21 how should my json report be named?
Im using this format: #{deviceName}report#{time}.json
You can skip Time variable if you want. ^_^
@mathias21 is this merged ?? as i'm in need of this now.... we have all set on the our private cloud to run the regression ...
Hi @saikrishna321
This is merged for both projects needed, this one and masterthought/cucumber-reporting.
You can configure the report for parallel executions from Jenkins. Let me know if something is wrong.
Gr. Jorge.
@mathias21 sure, i will give a try with the above comments.
I think you need to add the parallel option back into the Jenkins project - I took it out as dep points to released version of cucumber-reporting thus broken constructor. Change the dep in the Pom to use your local build of cucumber reports
I will release probably tonight also
Thanks @kingsleyh. Im quite busy right now, so @saikrishna321 you have to fly alone now. ^_^
You can take them from my repos, https://github.com/mathias21/cucumber-reporting https://github.com/mathias21/jenkins-cucumber-jvm-reports-plugin-java
Any news on this? I tried building an HPI file from the code that is currently in the jenkins plugin repo and made sure to use my local copy of the cucumber-reporting project as that one has the parallel fix. (I got it straight from the repo). However I still could not get it to work. The plugin would never install properly to Jenkins...
Hi @kingsleyh
Do I have to change something to make this work?
Thank you.
@mathias21 @kingsleyh Is this working? Currently I am trying to display parallel reports on jenkins, I have installed cucumber-reports-0.0.24 and selected the parallel testing box. When running cucumber the json files are getting created, eg ie-8-windows-7report.json but when I look at the cucumber reports it is just one report with all the features mutliplied by how many browsers I ran and there is no way of knowing when feature links to which browser.
Hi @nick-haworth Not sure about the state of master @kingsleyh. In my repo, you should set the parallel variable to true or use it within the jenkins plugin I modified, also merged with @kingsleyh master. If you are using ReportBuilder, then you can use this Constructor:
public ReportBuilder(List<String> jsonReports, File reportDirectory, String pluginUrlPath, String buildNumber, String buildProject, boolean skippedFails, boolean undefinedFails, boolean flashCharts, boolean runWithJenkins, boolean artifactsEnabled, String artifactConfig, boolean highCharts, boolean parallelTesting) throws Exception;
and set the latest boolean parameter (parallelTesting) to true.
I probably just merged into master with it disabled - maybe it needs a config checkbox in the options so it can be enabled from the Jenkins plugin if desired
Sent from my iPhone
On 12 Mar 2015, at 15:07, Jorge notifications@github.com wrote:
Hi @nick-haworth Not sure about the state of master @kingsleyh. In my repo, you should set the parallel variable to true or use it within the jenkins plugin I modified, also merged with @kingsleyh master. If you are using ReportBuilder, then you can use this Constructor:
public ReportBuilder(List
jsonReports, File reportDirectory, String pluginUrlPath, String buildNumber, String buildProject, boolean skippedFails, boolean undefinedFails, boolean flashCharts, boolean runWithJenkins, boolean artifactsEnabled, String artifactConfig, boolean highCharts, boolean parallelTesting) throws Exception; and set the latest boolean parameter (parallelTesting) to true. — Reply to this email directly or view it on GitHub.
@kingsleyh I also modified the jenkins-cucumber-jvm-reports-plugin-java to setup this. A new checkbox is added there to select parallel testing or not. If I remind correctly, already merged with your master.
Ok should be in the latest release then unless it's disabled - I vaguely remember it being disabled
Sent from my iPhone
On 12 Mar 2015, at 18:07, Jorge notifications@github.com wrote:
@kingsleyh I also modified the jenkins-cucumber-jvm-reports-plugin-java to setup this. A new checkbox is added there to select parallel testing or not. If I remind correctly, already merged with your master.
— Reply to this email directly or view it on GitHub.
@kingsleyh has this been merged with master for handling parallel json files?
I think so
Sent from my iPhone
On 26 Mar 2015, at 06:01, saikrishna321 notifications@github.com wrote:
@kingsleyh has this been merged with master for handling parallel json files?
— Reply to this email directly or view it on GitHub.
Such issues have been recently fixed by big refactor in version 1.00
After setting Jenkins to run a script to test same feature files over multiple devices, the generated feature reports are using the same name, so only the latest one is surviving. So you can see the different report rows inside the Cucumber Report, but when you try to access to the detailed html per feature, you always see the same one.
The problem comes from ReportBuilder in Cucumber-Reporting library. It is indexing using the feature name. It was not expected to have more than one device at the same time generating Jsons with the same feature.