tfhq / googletest

Automatically exported from code.google.com/p/googletest
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

XML output + repeating tests #426

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
IMPORTANT NOTE: PLEASE send issues or requests to
http://groups.google.com/group/googletestframework *instead of here*.
This issue tracker is NOT regularly monitored.

If you really need to create a new issue, please provide the information
asked for below.

What steps will reproduce the problem?
1. make a test that fails on every first call and passes next calls
2. run with parameters --gtest_repeat=10 --gtest_output=xml:test.xml
3. check test.xml
4. run with parameters --gtest_repeat=10 --gtest_break_on_failure 
--gtest_output=xml:test.xml

What is the expected output? What do you see instead?

text.xml should contain information about failed test at least when 
"--gtest_break_on_failure" is not specified
Instead text xml contains only last result - test passed. That's not very 
useful.
Alternatively I tried to use --gtest_break_on_failure in order to get the first 
fail, but I guess the break is meant for debuggers, which means that it doesn't 
reach the point where xml is generated

What version of Google Test are you using? On what operating system?
1.6, Linux, Centos 5 32 bit

Please provide any additional information below, such as a code snippet.

This is one way to test it:

#include <sys/time.h>

TEST(testCase, variableTestResults) {
  struct timeval  tv;
  gettimeofday(&tv, NULL);
  ASSERT_TRUE((tv.tv_usec%2) == 0);
}

I think there should be an option to choose what to output to xml file:
1. all runs of test suites - which means that every test suit element will 
appear N times in xml file (will be huge if you run it 1 000 000 times)
2. summary xml which includes number of passes/fails for each <testcase> 
element.

First might be redundant so adding number of passes/fails in xml file would be 
enough. Of course xml file must be generated after all runs are finished.

I'm not sure if this should be couted as bug or a feature request. I hope it 
will be considered.

Original issue reported on code.google.com by nicks...@gmail.com on 29 Mar 2013 at 12:36

GoogleCodeExporter commented 9 years ago
Of course this is a bug, test result must tell me how many ran and how many 
passed/failed, or the whole repeat feature is meaningless.
Any plans of fixing this?

Original comment by yuval.di...@gmail.com on 8 Jul 2015 at 11:44