Closed GoogleCodeExporter closed 8 years ago
Excellent plugin, thank you very much. I have tried the version 0.1 and it
parses the
failures / success correctly (except for the skipped tests, but that is a
limitation
from the nunit/hudson side of things).
However for our tests it doesn't parse the Error Message and Stacktrace yet
(all our
tests are implemented using the latest mbunit). I'll send you the details in
the
email.
once again thank you for this plugin!
Original comment by lau...@gmail.com
on 9 Mar 2009 at 7:25
I've seen the xml report file you sent me.
The quick test I've maid was pushing me to think that all the unit test report
produced by Gallio was the same. But in fact there is some little difference
(I've
just mais the test with nunit).
Tonight, I've reproduced the problem with all the xml files used to do the unit
test
with mbunit. I know where is the problem (a modification in the xsl file) but
I'm not
enought good to correct it in 5 minutes.
I can't correct it tonight but hope to do it this week.
You can help if you have some knowledge in xsl or know someone who have some.
You can
even test to modify it. The file is in the hudson workspace juste following the
path
".\plugins\gallio\WEB-INF\classes\hudson\plugins\gallio\gallio-to-junit.xsl".
For the "skipped" tests it's a lack of the junit xml file format (not
documented at
all) which don't displayed them.
Original comment by pmios...@gmail.com
on 10 Mar 2009 at 12:14
You can find there the second version of the plugin which display now well the
errors
of a mbunit unit test.
To be frank, the presentation is not sexy but it does the job.
Try it and make a feedback...
Original comment by pmios...@gmail.com
on 10 Mar 2009 at 11:59
Attachments:
Thanks for this! I am using v0.2 and it works quite well for successes and
failures
but doesn't seem to display the error message or stacktraces when I click on the
failed test; it seems blank. I took a look at the XSL but was unable to see
anything
immediately wrong as I don't have much experience with it. I attached one such
failure testlog element, to see if you might have any ideas.
Thanks again for the great plugin so far!
Original comment by mroo...@gmail.com
on 3 Apr 2009 at 9:41
Attachments:
Here is a screen shot of what I see at the page for the test that I attached. I
figure it may help in debugging. Sorry for having to smudge the test name :)
Original comment by mroo...@gmail.com
on 3 Apr 2009 at 10:21
Attachments:
Just 2 questions to understand your problem...
- Have you sent me the whole xml file generated by the unit test tool? Because
the
file seems quite empty and there is no xml root. Without I can't do anything.
- Can you confirm that you really use Gallio and not just MbUnit and if it's the
case, which version. It could help beacause I have only tested withe the 3.0.5
version and there is a new one now (3.0.6)
Thanks
Original comment by pmios...@gmail.com
on 3 Apr 2009 at 10:53
Thanks for the quick reply pmiossec! I am definitely using Gallio, however
version
3.0.6. Like I mentioned it works 90% as the tests show up with the successes and
failures properly, but the error type / message / stacktrace does not show up.
The XML file I sent was just what I thought was the relevant part. Please find
attached the full XML file.
Original comment by mroo...@gmail.com
on 4 Apr 2009 at 12:05
Try to locate the file named "gallio-to-junit.xsl" in the gallio plugin folder
in
your hudson install folder (should be like
"[huson_workspace]\plugins\gallio\WEB-INF\classes\hudson\plugins\gallio") and
replace
it by the one attached in this message.
It must not be very beautiful but I have no time today to improve it.
I can't actually provide a new package because I have to add a unit test for
your
file, test the impact of the (quick and dirty) modification I made.
And finally, I must succeed to release the package into hudson repository. I
don't
actualy :(
Original comment by pmios...@gmail.com
on 4 Apr 2009 at 11:55
Attachments:
Thanks pmiossec, I will give it a try on Monday! I already located that file to
give
fixing it a try myself so I have no problem packing up a new plugin myself. By
the
way is there any reason for the gallio-to-junit.xsl.bak file that is there? It
seems
like maybe it is confusing to release that :)
Thanks again and good luck on getting it into the Hudson repo, I would love to
see it
there as it is very useful, surely for many other people too.
Original comment by mroo...@gmail.com
on 4 Apr 2009 at 6:23
All right pmiossec, I've tested it and have mixed results (but good overall :).
First, it shows every test twice. There are 23 tests and 3 failures, but with
the new
version it shows 46 tests and 6 failures (each test twice). However in good
news, the
error message and stacktrace works for the second result of each test. Ie the
failed
test list looks like: A, B, C, A, B, C, and the error output shows up correctly
on
the second set of failed tests but not the first. Hopefully that is just a
simple
fix. Thanks again for your help!
Original comment by mroo...@gmail.com
on 6 Apr 2009 at 5:49
As it turns out, the double counting was because both plugins were enabled, even
though I disabled one in Hudson. I actually had to uninstall the old plugin,
but now
it works just fine, thanks!
Original comment by mroo...@gmail.com
on 9 Apr 2009 at 9:44
Yes, I had a look 2 days ago for this problem and due to the file you sent me
and
was thinking it was such that sort of problem...
I'm happy you find it.
Original comment by pmios...@gmail.com
on 10 Apr 2009 at 7:12
Have you looked into submitting this to Hudson more officially? I think it
would be a
great addition and could help a lot of people!
Original comment by mroo...@gmail.com
on 10 Apr 2009 at 6:11
I try, I try!!! and a lot....for already almost a month :(
I have the write access in the svn repository.
All the source code is already archived by I can't succeed in releasing the
plugin (
with maven ).
I don't actually how to do it :(
Original comment by pmios...@gmail.com
on 11 Apr 2009 at 3:10
After a lot of efforts.... it should be done :)
The release seemed to work well.
Wait and see :)
Original comment by pmios...@gmail.com
on 13 Apr 2009 at 10:42
I see that I am getting updates for this via Hudson, which is awesome, thanks!
It
isn't linked to a wiki page though; is there any place that development is
happening
that I can follow, particularly being able to view a changelog? It would be
great to
know what kind of updates are coming in and how important it is that I install
them
and restart Hudson.
Original comment by mroo...@gmail.com
on 29 Apr 2009 at 12:08
Yeh, the plugin is published and I've created the wiki page.
For the moment, there is no differences between you're version and this one
published
(if I remember well ;) )
Original comment by pmios...@gmail.com
on 30 Apr 2009 at 9:08
So is there anything that you think Gallio could do to help people find the
Hudson
plugin?
Original comment by jeff.br...@gmail.com
on 29 Sep 2009 at 1:36
Hi
I'm having a problem when using the Hudson plugin with Gallio version 3.1. The
problem
is that I can't get the Gallio plugin to process any of the Gallio reports. I'm
pretty
sure I've pointed it at the correct log files but it claims that there are no
tests
(there's about 30 of them). The strange thing is that at home it works fine
with Gallio
3.0.6 but at work it doesn't seem to function with Gallio 3.1. Any ideas on how
to make
it work with Gallio 3.1?
Original comment by petrikva...@gmail.com
on 22 Oct 2009 at 8:05
This is probably due to changes in the report schema. It should be relatively
easy
to fix given the code of the Hudson plugin.
Original comment by jeff.br...@gmail.com
on 23 Oct 2009 at 6:21
Should this plugin still work with Hudson 1.329?
Gallio executes my unit tests and generates a mbunit-result.xml file which looks
correct however the generated junit-result.xml seems to be invalid and only
lists 18
test out of the 227 I have. Even the ones it lists seem wrong:
<file>D:\Hudson\jobs\OcadoTools_MS\workspace\temporary-junit-reports\TEST-.xml</
file>
<name></name>
<duration>29.986736</duration>
<cases>
<case>
<duration>7.7514496</duration>
<className></className>
<testName>Alert</testName>
<skipped>false</skipped>
<failedSince>0</failedSince>
</case>
<case>
<duration>0.2690693</duration>
<className></className>
<testName>CommandLine</testName>
<skipped>false</skipped>
<failedSince>0</failedSince>
</case>
Any ideas?
Let me know if you need any other info.
Help is much appreciated,
James
Original comment by JGral...@gmail.com
on 27 Oct 2009 at 1:01
Yeah, the problem should come from the new format of the version 3.1 of Mbunit.
I should fix that but I can't find some time to do that.
I will try to remotivate myself due to the fact that you're the third who ask
for that.
Eventually, if someone have some knowledge in xslt (and time), I'll be pleased
to
explain how to correct.
Nevertheless, a post of the xml file generated by mbunit could help me.
Original comment by pmios...@gmail.com
on 29 Oct 2009 at 10:44
Count one more....
Thanks!
Original comment by jamonl...@gmail.com
on 1 Nov 2009 at 10:04
Count one more as well. We run some 2000 tests, many of which are combinatorial
tests
plus we have dynamically generated tests and test suites. Currently the plugin
reports that we have 63 tests. I am attaching a zip archive - Debug.zip, which
contains the Gallio report in HTML and XML formats. Both relate to the same
unit test
run.
Original comment by mark.kha...@gmail.com
on 3 Nov 2009 at 12:38
Attachments:
And, BTW, thank you very much.
Original comment by mark.kha...@gmail.com
on 3 Nov 2009 at 12:39
I had a look yesterday and in fact de xml report file format for Gallio had
changed
a lot with the 3.1 version.
I hope I could have a file you could test tonight.
Original comment by pmios...@gmail.com
on 3 Nov 2009 at 3:14
For those of you that use Gallio version 3.1 (and only this version) with
hudson,
could you test this file and make me some returns?
The file should replace the file located in the gallio plugin directory of
hudson.
The path should be that way (but you can search by yourself the file named
"gallio-to-junit.xsl")
[Your_Hudson_Worspace_directory]\plugins\gallio\WEB-INF\classes\hudson\plugins\g
allio\gallio-to-junit.xsl
If it resolve your problems, I will update the plugin (if I could remember how
to do
that :) )
Original comment by pmios...@gmail.com
on 3 Nov 2009 at 10:24
Attachments:
Hi there,
Just wanted to let you know that this new file fixed the issue in my setup. I
don't
know if it will work for everyone else, but I finally got a beautiful test
result
graph here.
Thanks for both this and the wonderful plugin!
Original comment by havremun...@gmail.com
on 4 Nov 2009 at 8:06
What about support for Gallio 3.2? *duck*
Original comment by straufl@gmail.com
on 10 Nov 2009 at 12:16
Hi!
Is there someone except havremunken which have tested the file?
Does it work and solve your problems or not?
Because I surely will merge the file with the existing one to publish a new
version
of the plugin in order to permit at all the user to benefit of the
correction....
Let me know!
Original comment by pmios...@gmail.com
on 14 Nov 2009 at 12:00
@straufl, Gallio v3.2's report format has not changed since v3.1 yet, so it
should
all work.
Original comment by jeff.br...@gmail.com
on 16 Nov 2009 at 4:11
Though running 42 tests I only see one successful one in my Hudson result. It
only
contains the running time of one test, but no name, no namespace etc. Changing
gallio-to-junit.xsl didn't change anything.
The xml file produced by Gallio (during usage of the wildcard feature *.dll)
seems to
contain all the necessary information, but it seems that it doesn't get parsed
the
right way :(
Original comment by straufl@gmail.com
on 16 Nov 2009 at 7:11
Yes, the XSLT needs to be improved so that it can handle all of the necessary
features. I sent an email to the author with additional details.
Original comment by jeff.br...@gmail.com
on 16 Nov 2009 at 7:24
Hiya
Is there any progress on this issue? I'd really like to use 3.1 on our build
server
but we're currently blocked because of this issue.
Also is there a way of parsing multiple XML files? One of our projects consists
of
multiple solutions, each of which generate their own XML file. I've tried
setting up
the configuration like so:
**/modules/utils/bin/reports/test-report-*.xml,**/modules/core/bin/reports/test-
report-*.xml
but that doesn't seem to work. Any suggestions?
Thanks
Patrick
Original comment by petrikva...@gmail.com
on 1 Dec 2009 at 4:53
@petrik: Maybe you should open up a separate issue for the multiple files
problem.
@jeff: The author seems to be busy. Could you maybe post these details, so
somebody
else might give it a try?
Thanks,
Flo
Original comment by straufl@gmail.com
on 2 Feb 2010 at 10:09
@pmiossec: Just to let you know that I've tried the modified
gallio_to_junit.xsl and
it worked for me too.
Original comment by paul@therendellhouse.co.uk
on 10 Feb 2010 at 7:27
@paul: Which Galio version did you use?
Original comment by straufl@gmail.com
on 11 Feb 2010 at 8:12
The most recent gallio-to-junit.xsl still does not work. Attached are sample
condensed HTML and the respective XML mb-unit reports for reference. These
represent
actual results of actual unit tests that we run - many of them are dynamic and
combinatorial.
Note, that mb-unit themselves do not construct the HTML correctly, see the issue
http://code.google.com/p/mb-unit/issues/detail?id=643 for details.
Original comment by mark.kha...@gmail.com
on 24 Mar 2010 at 8:12
Attachments:
Forgot to mention, we use Gallio 3.1
Original comment by mark.kha...@gmail.com
on 24 Mar 2010 at 8:16
Hey all,
I've got an _extremely_ minimal version of a working XSL - I've only tested it
on my
setup (Gallio 3.1 build 397 and Hudson 1.349), and only on a simple test setup
(one
fixture, a dozen tests). It still needs a lot of work - I've included it here
as it's
slightly better than having nothing at all =)
The biggest obstacle I'm running into is that I don't know what the jUnit xml
output
is supposed to look like. Does anyone know where I can find a XSD for that
format, or
at least a non-trivial example? My googling has completely failed me on this =(
Original comment by brian.de...@gmail.com
on 30 Mar 2010 at 10:11
Attachments:
When I looked to find some junit documentation when I begun to write the
plugin, I
was enable to find somthing too :( It seems that noone have a documentation.
PS : 1. I'm sorry but I have no time to continue to work on this plugin
(especially
the xslt file).
If you succeed in doing a good xslt file, perhaps you can contact Gregory
Boissinot
which work on the xUnit and Mbunit plugin ( you'll we find it's mail here
http://wiki.hudson-ci.org/display/HUDSON/Gallio+Plugin )
2. jeff brown (from the Gallio project) write me to make me some advice and
correction on the xslt file. If you need these infos, there it is (sorry for the
length!!!)
--------------------------------------------------------------------------------
--
The core of this seems to be gallio-to-junit.xsl. We could actually ship this
as
part of Gallio as a new report format. So you'd run your tests with
"/report-type:JUnit" to get a JUnit-style report.
I think this could be useful for lots of people. Would you be willing to take
this
XSLT the final mile and get it shipped with Gallio in future releases? Mainly
this
would require a few design changes and some tests.
The current code is not very robust. It assumes that the tests are laid out in
a
very particular way and that stack traces are formatted just so. Unfortunately
you
will see slightly different output from different test frameworks or when using
special features of some test frameworks. Mapping all possible structures
nicely
onto JUnit is a considerable challenge.
Complication #1. Tests can be arbitrarily nested.
This code makes an incorrect assumption about test case layout.
<testsuites>
<xsl:for-each
select="a:testStepRun/a:children/a:testStepRun/a:children/a:testStepRun">
<testsuite skipped="." failures="." errors=".">
The top-level test step run is always the root. The second-level test step run
is
always a test file (assembly). Beyond that, we don't know, exactly what we're
looking at. It could be a fixture, a test case, a nested test step, or a
data-driven
instance of a fixture or test case, or something else. So we can't assume that
the
third-level test step runs represent fixtures or that the fourth-level step runs
represent test cases even though that's true for MbUnit when no fancy features
are
being used.
The way out of this mess is to use some special attributes that Gallio provides
about
test step runs.
@isTestCase : Specifies whether this test step run a test case (like a test
method).
@isPrimary: Specifies whether this test step run is the outermost run of a given
fixture or test case. For data-driven tests like [Row] tests, there will be a
primary run that contains non-primary runs for each row.
Interesting combinations:
@isTestCase && @isPrimary : This is a test case like a test method.
! @isTestCase && @isPrimary : This is a test container of some kind like a test
suite
or test fixture.
@isTestCase && ! @isPrimary : This is a particular instance of a data-driven
test case.
! @isTestCase && ! @isPrimary: This is a particular instance of a data-driven
test
container or some kind.
So you need logic like this:
<xsl:template match="a:testStepRun">
<xsl:choice>
<xsl:when test="@isTestCase">
<testcase>
<!-- emit some interesting stuff... -->
<!-- remember to recurse! test cases can contain other runs... don't
know how this maps into JUnit, might take some imagination -->
<xsl:apply-templates select="a:children" />
</testcase>
</xsl:when>
<xsl:otherwise>
<testsuite>
<!-- emit some interesting stuff... -->
<!-- remember to recurse! test containers can contain other runs...
don't know how this maps into JUnit, might take some imagination -->
<xsl:apply-templates select="a:children" />
</testsuite>
</xsl:otherwise>
</xsl:choice>
</xsl:template>
Complication #2. Test output is just formatted text.
I'll admit. Grabbing a raw stack trace from Gallio is HARD. There's a good
reason
for it too. Tests can emit multiple stack traces! To help the user figure this
stuff out, we format stack traces and failure messages in all sorts of fancy
ways.
However, it does mean that tools that expect that one test run produces one
stack
trace are out of luck. Gallio violates their underlying assumptions.
Unfortunately this kind of code can't work:
<xsl:for-each
select="a:testLog/a:streams/a:stream/a:body/a:contents/a:section/a:contents/a:ma
rker/a:contents/a:marker"
>
<xsl:if test="@class = 'StackTrace'">
<xsl:value-of select="a:contents/a:text"/>
<xsl:for-each select="a:contents/a:marker">
<xsl:value-of select="a:contents/a:text"/>
</xsl:for-each>
</xsl:if>
</xsl:for-each>
So here's what you need to do: format the test log output in some nice way and
just
stuff it in the JUnit test case message attribute.
...
<testcase>
<xsl:attribute name="message">
<xsl:apply-templates select="g:testLog" mode="results">
</xsl:attribute>
</testcase>
...
</xsl:template>
<!-- Formats the test log as text. This is copied from
Gallio-Report.txt-common.xslt. You might want to customize it to pretty-print
certain log streams or marker classes differently if you like. --->
<xsl:template match="g:testLog" mode="results">
<xsl:apply-templates select="g:streams/g:stream/g:body"/>
</xsl:template>
<xsl:template match="g:body">
<xsl:apply-templates select="g:contents" mode="block" />
</xsl:template>
<xsl:template match="g:contents" mode="block">
<xsl:variable name="text">
<xsl:apply-templates select="." mode="inline" />
</xsl:variable>
<xsl:call-template name="indent">
<xsl:with-param name="text" select="$text"/>
</xsl:call-template>
</xsl:template>
<xsl:template match="g:contents" mode="inline">
<xsl:apply-templates select="child::node()[self::g:text or self::g:section or
self::g:embed or self::g:marker]" />
</xsl:template>
<xsl:template match="g:text">
<xsl:value-of select="text()"/>
</xsl:template>
<xsl:template match="g:section">
<xsl:text>
</xsl:text>
<xsl:value-of select="@name" />
<xsl:text>
</xsl:text>
<xsl:apply-templates select="g:contents" mode="block" />
</xsl:template>
<xsl:template match="g:marker">
<xsl:apply-templates select="g:contents" mode="inline" />
</xsl:template>
<xsl:template match="g:embed">
<xsl:text>
[Attachment: </xsl:text>
<xsl:value-of select="@attachmentName"/>
<xsl:text>]
</xsl:text>
</xsl:template>
Original comment by pmios...@gmail.com
on 1 Apr 2010 at 10:37
I cannot help the development effort (alas), but I will certainly be awaiting
for the
new report type very anxiously.
Original comment by mark.kha...@gmail.com
on 3 Apr 2010 at 5:14
@pmiossec: No need to apologize - thanks for building this thing in the first
place =)
Unfortunately, I don't have spare time to put towards this right now. My own
requirements are fairly limited, so I can't justify taking this much farther
than
solving my own problems at the moment =( The best I can do is promise that if I
do
make any other enhancements, I'll post them here.
- Brian
Original comment by brian.de...@gmail.com
on 5 Apr 2010 at 1:37
[deleted comment]
It now works on my machine running Hudson ver. 1.352 and Hudson Gallio plugin
0.70
and Hudson xUnit plugin 0.6.1 (might be required as well) with the attached
xslt
file (can't remember where I got it from)
Original comment by straufl@gmail.com
on 19 May 2010 at 1:08
Attachments:
Hudson Gallio plugin (from 0.70) extends xUnit Hudson plugin (0.6.1).
You need to have both plugins in your $HUDSON_HOME/plugins directory.
You can find the Gallio stylesheet in the Gallio Hudson plugin source code.
http://fisheye.hudson-
ci.org/browse/Hudson/trunk/hudson/plugins/gallio/src/main/resources/hudson/plugi
ns
/gallio
In the next release, an option should be available for download the stylesheet.
Original comment by gregory.boissinot
on 20 May 2010 at 7:49
We have fixed the .XSL file in version 0.70 of the plugin to allow it to read
reports
from Gallio v3.1.
The file was also tweaked to have Hudson show the exception message as well as
the
exception type for a failing test. Before, it would just show the exception
type.
I have attached the modified file to this comment. I hope this helps.
Original comment by vprud...@gmail.com
on 28 May 2010 at 2:08
Attachments:
The xsl transform did not show the individual tests in junit xml file. Only the
tests with variants showed up. This was probably an issue with Gallio version
3.1.
I will commit the fix to Hudson svn (trunk) as soon as I get the commit access
there.
Attached you will find the fixed xsl file.
Best Regards,
Tommi Palomäki
Cybercom
Original comment by kom...@gmail.com
on 5 Jul 2010 at 9:33
Attachments:
I'd just like to thank Tommi for providing the xsl file here - my team has
spent quite a bit of time trying to figure out why we were getting the error
"The plugin hasn't been performed correctly: None of the test reports contained
any result" in our output from xUnit via the Gallio plugin. Using the supplied
"gallio-to-junit.xsl" file attached above fixes the issue.
NOTE: for those using Gallio 3.2 Alpha it works as well.
Cheers,
Zac Seth
Original comment by zac.s...@gmail.com
on 25 Aug 2010 at 1:06
Is this now part of the plugin? I can see that the default stylesheet name
(when you install the plugin) has changed from gallio-to-junit.xsl to
gallio-1.0-to-junit-1.0.xsl...
Original comment by jamonl...@gmail.com
on 2 Sep 2010 at 11:21
Original issue reported on code.google.com by
pmios...@gmail.com
on 4 Mar 2009 at 11:02