anticoders / gagarin

Another testing framework for your meteor apps.
gagarin.meteor.com
MIT License
150 stars 21 forks source link

Reporter output should not be affected by Gagarin #165

Open Kostanos opened 8 years ago

Kostanos commented 8 years ago

I'm using gagarin gagarin -T 15000 -t 15000 -m --reporter xunit > xunit.xml

When everything is ok, xunit.xml is with valid format

But if there is error, I got invalid XML.

Ex:

<testsuite name="Mocha Tests" tests="13" failures="1" errors="1" skipped="0" timestamp="Mon, 29 Feb 2016 23:27:28 GMT" time="68.916">
<testcase classname="Server gagarin test suite" name="check gagarin and meteor" time="0.009"/>
<testcase classname="Server fixtures (need to be enabled for tests)" name="we need fixtures for tests" time="0.014"/>
<testcase classname="Server push notifications test suite" name="Check onUpdateOrderStatus notifications" time="0.025"/>
<testcase classname="Server push notifications test suite" name="Check onUnattendedOrder notification" time="0.042"/>
<testcase classname="Server meals collections test suite" name="Check Valid Extras" time="0.024"/>
<testcase classname="Server meals collections test suite" name="Check Invalid Extras, mealId and title" time="0.012"/>
<testcase classname="Server meals collections test suite" name="Check Invalid Extras, no mealId and no title" time="0.007"/>
<testcase classname="Server meals collections test suite" name="Check Invalid Extras, no mealId with priceFromMeal" time="0.008"/>
<testcase classname="Frontend browser test suite" name="should be able to emulate tablet device" time="0.078"/>
<testcase classname="Frontend Gagarin test suite" name="check gagarin and meteor in client" time="0.099"/>
<testcase classname="Frontend login and logout test suite" name="login with no manager and logout" time="1.141"/>
<testcase classname="Frontend login and logout test suite" name="login with manager and go to user profile page" time="5.241"><failure><![CDATA[I have been waiting for 5000 ms until element .brand-logo.text-center:nth-of-type(1) is present, but it did not happen.

</failure></testcase>
<testcase classname="Frontend new meal test suite" name="create a new meal without extras" time="1.743"/>
</testsuite>

It seems CDATA is not closed

Kostanos commented 8 years ago

I found the problem, it is sed that I run after, to clean up the report.

gagarin doesn't make it clean.

This is what I run, and it doesn't work, so any help is welcome.

sed -n '/testsuite name/,$p' xunit.xml.orig > xunit.xml
sed -i 's/^[^<]*//' xunit.xml
Kostanos commented 8 years ago

This is what works for me as workaround, for those who has the same issue:

sed 's/.*<testsuite /<testsuite /' xunit.xml.orig | sed -n '/testsuite name/,$p' > xunit.xml

I still this task still should be opened, gagarin should suppress all output when generate xunit report.

apendua commented 8 years ago

@Kostanos This is probably a good idea. What if we had an option for that? So instead of making a guess gagarin could be instructed not to produce any additional output.

Kostanos commented 8 years ago

@apendua alternative is to explicitly set the report output file

example: gagarin --reporter xunit --report_output <report filename>