rajdeepv / parallel_calabash

Helps running calabash ios/android tests in parallel
MIT License
40 stars 19 forks source link

Multiple devices - single report #13

Open mikevandre opened 9 years ago

mikevandre commented 9 years ago

I'm very excited to have found this project. I've noticed however that with the html results I am experiencing a few issues. With two devices and --group-by-scenarios I am only getting half of the results. Running without this option I am getting half of the results and the other half is not outputting properly (see the screenshot attached) capture

rajdeepv commented 9 years ago

can you please share the complete command you are using?

mikevandre commented 9 years ago

parallel_calabash -a C:\src\test\test\trunk\AutomatedTests\apks\Teamcity\Test-6280.apk -o '-p android --format html -o results.html' features/

the -p command is used to tell the tests this is a phone versus a tablet

On Thu, Aug 6, 2015 at 11:01 AM, Rajdeep notifications@github.com wrote:

can you please share the complete command you are using?

— Reply to this email directly or view it on GitHub https://github.com/rajdeepv/parallel_calabash/issues/13#issuecomment-128404552 .

rajdeepv commented 9 years ago

Please use this: parallelcalabash -a C:\src\test\test\trunk\AutomatedTests\apks\Teamcity\Test-6280.apk -o '-p android --format html -o results%TEST_PROCESS_NUMBER%.html' features/

mikevandre commented 9 years ago

That will result in multiple files then correct? Is there a way to create a consolidated report?

On Thu, Aug 6, 2015 at 12:50 PM, Rajdeep notifications@github.com wrote:

Please use this: parallel_calabash -a C:\src\test\test\trunk\AutomatedTests\apks\Teamcity\Test-6280.apk -o '-p android --format html -o results%TEST_PROCESS_NUMBER%.html' features/

— Reply to this email directly or view it on GitHub https://github.com/rajdeepv/parallel_calabash/issues/13#issuecomment-128438621 .

rajdeepv commented 9 years ago

as of now there is no functionality to get consolidated report

mikevandre commented 9 years ago

I tried your suggestion and I am still having the output placed in a single file with formatting issues...

parallel_calabash -a C:\src\MCC\MCC\trunk\AutomatedTests\apks\Teamcity\MCCNotes-6308.apk -o '-p android --format html -o TesResults%TEST_PROCESS_NUMBER%.html' features/

this is only creating a single TestResults.html file. It looks like the %TEST_PROCESS_NUMBER% is not populated?

Mike

On Fri, Aug 7, 2015 at 1:11 AM, Rajdeep notifications@github.com wrote:

as of now there is no functionality to get consolidated report

— Reply to this email directly or view it on GitHub https://github.com/rajdeepv/parallel_calabash/issues/13#issuecomment-128600525 .

rajdeepv commented 9 years ago

can you please try this with cucumber.yml default: --format html --out reports/TesResults _<%= ENV['TEST_PROCESS_NUMBER']%>.html --format pretty

nazeercool commented 8 years ago

Hi everyone ... Currently am doing calabash-android ... I want to run the test on multiple devices and I need the output in single html format .... Currently I run on two devices I need to open two shells and I need to type the command for each serail number and if I give different report name I get two reports and if I give same file name am receiving only one report with the same test cases but I don't find the another test case which has run for another device ...

Please I need your help

ghost commented 7 years ago

Might be a little late but if you want multiple reports in a single html file look into using allure-cucumber. https://github.com/allure-framework/allure-cucumber run your calabash command with '--format AllureCucumber::Formatter --out xml/tests' after you install it. Then while all xml files are in the same folder run 'allure generate directory-with-results/ -o directory-with-report' http://wiki.qatools.ru/display/AL/Allure+Commandline & https://github.com/allure-framework/allure1/releases/tag/allure-core-1.5.0 After you set it up. Worked for me although I couldn't seem to tell the devices apart on the report which is the only downside. Resorted in just using http://blog.lesspainful.com/2013/03/15/Testing-Multiple-Android-Apps/ with Allure. Also found parallel failing simple scenarios after a long period of time using multiple devices.