Incorrent xml output if I use the dataProvider in tests

[bayandin]

I think this issue continues #10.

If I use dataProvider in tests I get incorrect xml output (it different from standard phpunit). The difference is that phpunit output has another <testsuite> tag and testsuite has a different attribute name. See example below.

• PHPUnit 3.7.13 from Composer,
• Paratest 0.4.1 or dev-master (545c4a7)

Code example:

<?php

class simpleTest extends PHPUnit_Framework_TestCase
{
    public function provider()
    {
        return array(
            array('foo'),
            array('bar'),
        );
    }

    /**
     * @dataProvider provider
     */
    public function test1()
    {
        $this->assertTrue(false);
    }
}

vendor/bin/phpunit --log-junit=phpunit-junit.xml SimpleTest.php

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="SimpleTest" file="/home/a.bayandin/PhpstormProjects/Online4/tests/SimpleTest.php" tests="2" assertions="2" failures="2" errors="0" time="0.001455">
    <testsuite name="SimpleTest::test1" tests="2" assertions="2" failures="2" errors="0" time="0.001455">
      <testcase name="test1 with data set #0" assertions="1" time="0.001205">
        <failure type="PHPUnit_Framework_ExpectationFailedException">SimpleTest::test1 with data set #0 ('foo')
Failed asserting that false is true.

/home/a.bayandin/PhpstormProjects/Online4/tests/SimpleTest.php:18
</failure>
      </testcase>
      <testcase name="test1 with data set #1" assertions="1" time="0.000250">
        <failure type="PHPUnit_Framework_ExpectationFailedException">SimpleTest::test1 with data set #1 ('bar')
Failed asserting that false is true.

/home/a.bayandin/PhpstormProjects/Online4/tests/SimpleTest.php:18
</failure>
      </testcase>
    </testsuite>
  </testsuite>
</testsuites>

vendor/bin/paratest -p 1 --phpunit=vendor/bin/phpunit --log-junit=paratest-junit.xml SimpleTest.php

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="" tests="2" assertions="2" failures="2" errors="0" time="0.001525" file="">
    <testcase name="test1 with data set #0" class="" file="" line="" assertions="1" time="0.001247">
      <failure type="PHPUnit_Framework_ExpectationFailedException">SimpleTest::test1 with data set #0 ('foo')
Failed asserting that false is true.

/home/a.bayandin/PhpstormProjects/Online4/tests/SimpleTest.php:18
</failure>
    </testcase>
    <testcase name="test1 with data set #1" class="" file="" line="" assertions="1" time="0.000278">
      <failure type="PHPUnit_Framework_ExpectationFailedException">SimpleTest::test1 with data set #1 ('bar')
Failed asserting that false is true.

/home/a.bayandin/PhpstormProjects/Online4/tests/SimpleTest.php:18
</failure>
    </testcase>
  </testsuite>
</testsuites>

[brianium]

Awesome thanks for bringing this up. I'll take a look into this. Any issue noticed with running against a directory?

[bayandin]

No, I haven't noticed anything unusual yet.

[vavr]

has any ideas when this bug will be fixed?

[PieterDC]

I can confirm this bug still exists with version 0.12.0 of paratest.

[gitnik]

Also experiencing this

[tarjei]

Yes, I'd like to vote for this bug as well. Any tips wrt. when this bug will be fixed?

[tarjei]

Or, how it could be fixed.

[julianseeger]

A partly fix is on the way with PR #185

[julianseeger]

About the "testsuite" elements containing child "testsuite" elements, @drefixs brought up some infos here: https://github.com/brianium/paratest/pull/185#issuecomment-156999270 As phpunit tries to produce junit-like logs and there is no trace that junit logs do or are allowed to contain nested "testsuite"-elements, I agree with @drefixs that phpunit is doing it wrong.

Any opinions on this? Trying to get a fix in all maintained phpunit versions (I think the best we would get is a fix in the next major version because of BC) / adding a flag to produce "wrong" junit logs / ... Or is this even the core problem of this thread or was it more about empty names etc.?

[tarjei]

Jenkins has a separate parser for junit xml logs from PHPUnit. If we can use the normal one without problems then IMHO that is a win.

2015-11-18 0:08 GMT+01:00 Julian Seeger notifications@github.com:

About the "testsuite" elements containing child "testsuite" elements, @drefixs https://github.com/drefixs brought up some infos here: #185 (comment) https://github.com/brianium/paratest/pull/185#issuecomment-156999270 As phpunit tries to produce junit-like logs and there is no trace that junit logs do or are allowed to contain nested "testsuite"-elements, I agree with @drefixs https://github.com/drefixs that phpunit is doing it wrong.

Any opinions on this? Trying to get a fix in all maintained phpunit versions (I think the best we would get is a fix in the next major version because of BC) / adding a flag to produce "wrong" junit logs / ... Or is this even the core problem of this thread or was it more about empty names etc.?

— Reply to this email directly or view it on GitHub https://github.com/brianium/paratest/issues/19#issuecomment-157539249.

---

Tarjei Huse Mobil: 920 63 413

[dhopkalo]

@tarjei just see: https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd (jenkins shema) https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd (common shema)

Are you see any testsuite into testsuite ? jenkins can parse it but other CI system can't. Now paratest has my fix for this problem #185 you can try get last version, and test it now.

[julianseeger]

Seems to be fixed. If anyone disagrees, please open a new issue with new infos what's good and whats missing.