'make check' reports several failures

[GoogleCodeExporter]

What steps will reproduce the problem?

Follow steps from README files:
  $ ${SRCDIR}/configure  # Standard GNU configure script, --help for more info
  $ make  # Standard makefile following GNU conventions
  $ make check  # Builds and runs all tests - all should pass

What is the expected output? What do you see instead?
Expected output: Builds and runs all tests - all should pass
What I got was:

'make check' fails with following messages:

=====================

FAIL: Tests using the GTEST_BREAK_ON_FAILURE environment variable.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test/gtest_break_on_failure_unittest.py", line 148, in testEnvVar
    expect_seg_fault=1)
  File "./test/gtest_break_on_failure_unittest.py", line 131, in RunAndVerify
    self.assert_(has_seg_fault == expect_seg_fault, msg)
AssertionError: when GTEST_BREAK_ON_FAILURE=1, an assertion failure in
"/home/toni/work/gtest-1.0.0/test/gtest_break_on_failure_unittest_" should
cause a seg-fault.

======================================================================
FAIL: Tests using the --gtest_break_on_failure flag.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test/gtest_break_on_failure_unittest.py", line 158, in testFlag
    expect_seg_fault=1)
  File "./test/gtest_break_on_failure_unittest.py", line 131, in RunAndVerify
    self.assert_(has_seg_fault == expect_seg_fault, msg)
AssertionError: when GTEST_BREAK_ON_FAILURE is not set, an assertion
failure in
"/home/toni/work/gtest-1.0.0/test/gtest_break_on_failure_unittest_
--gtest_break_on_failure" should cause a seg-fault.

======================================================================
FAIL: Tests that the flag overrides the environment variable.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test/gtest_break_on_failure_unittest.py", line 168, in
testFlagOverridesEnvVar
    expect_seg_fault=1)
  File "./test/gtest_break_on_failure_unittest.py", line 131, in RunAndVerify
    self.assert_(has_seg_fault == expect_seg_fault, msg)
AssertionError: when GTEST_BREAK_ON_FAILURE=0, an assertion failure in
"/home/toni/work/gtest-1.0.0/test/gtest_break_on_failure_unittest_
--gtest_break_on_failure" should cause a seg-fault.

----------------------------------------------------------------------
Ran 4 tests in 0.178s

FAILED (failures=3)
FAIL: test/gtest_break_on_failure_unittest.py
 ...
3 of 26 tests failed
Please report to googletestframework@googlegroups.com
=====================================================
make[1]: *** [check-TESTS] Error 1
make[1]: Leaving directory `/home/toni/work/gtest-1.0.0'
make: *** [check-am] Error 2

What version of the product are you using? On what operating system?
My environment is Linux: Ubuntu 8.04:
uname -m = i686
uname -r = 2.6.24-16-server
uname -s = Linux
uname -v = #1 SMP Thu Apr 10 13:58:00 UTC 2008

python -V: Python 2.5.2

> gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v
--enable-languages=c,c++,fortran,objc,obj-c++,treelang --prefix=/usr
--enable-shared --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --enable-nls
--with-gxx-include-dir=/usr/include/c++/4.2 --program-suffix=-4.2
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc
--enable-mpfr --enable-targets=all --enable-checking=release
--build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.2.3 (Ubuntu 4.2.3-2ubuntu7)

Please provide any additional information below.

RunAndVerify function is mention several times but most probably problem is
in this short function from file "gtest_break_on_failure_unittest.py":

def Run(command):
  """Runs a command; returns 1 if it has a segmentation fault, or 0 otherwise.
  """

  return os.system(command) == signal.SIGSEGV

Statement:
  return os.system(command) == signal.SIGSEGV

never comes "true" because (if you want to test for SIGSEGV):
- signal.SIGSEGV has value 11
- os.system(command) returns 35584 in case that command was killed by
signal SIGSEGV.

Correct check would be (at least in my environment):
  ret = os.system(command)
  return (ret>>8) == (signal.SIGSEGV + 128)

Why this work is explained in man pages for system() and wait() in Python
and C:
=====================
system(     command)
    Execute the command (a string) in a subshell. This is implemented by
calling the Standard C function system(), and has the same limitations.
Changes to posix.environ, sys.stdin, etc. are not reflected in the
environment of the executed command.

    On Unix, the return value is the exit status of the process encoded in
the format specified for wait(). Note that POSIX does not specify the
meaning of the return value of the C system() function, so the return value
of the Python function is system-dependent.

wait(   )
    Wait for completion of a child process, and return a tuple containing
its pid and exit status indication: a 16-bit number, whose low byte is the
signal number that killed the process, and whose high byte is the exit
status (if the signal number is zero); the high bit of the low byte is set
if a core file was produced. Availability: Macintosh, Unix.
=====================

Similar result could be get if you start this sequence of command in bash:
=====================
> sleep 30 &
[1] 28937
> pid=$!
> echo $pid
28937
> kill -segv $pid
[1]+  Segmentation fault      sleep 30
> wait $pid
> echo $?
139
=====================

Original issue reported on code.google.com by anto.jur...@gmail.com on 9 Jul 2008 at 6:48

[GoogleCodeExporter]

BTW: file gtest_xml_output_unittest.py, function _TestXmlOutput() has part of 
code
similar to my proposal:

  def _TestXmlOutput(self, gtest_prog_name, expected_xml, expected_exit_code):
  ...
    command = ("%s %s=xml:%s &> /dev/null"
               % (gtest_prog_path, GTEST_OUTPUT_FLAG, xml_path))
    status = os.system(command)
    signal = status & 0xff
    self.assertEquals(0, signal,
                      "%s was killed by signal %d" % (gtest_prog_name, signal))
    exit_code = status >> 8
    self.assertEquals(expected_exit_code, exit_code,
                      "'%s' exited with code %s, which doesn't match "
                      "the expected exit code %s."
                      % (command, exit_code, expected_exit_code))
    ...

Original comment by anto.jur...@gmail.com on 9 Jul 2008 at 8:57

[GoogleCodeExporter]

This is fixed in trunk and release-1.0.1.

Original comment by shiq...@gmail.com on 9 Jul 2008 at 10:10

• Changed state: Fixed

[GoogleCodeExporter]

I am using Ubuntu 8.04 and googletest 1.0.1 and I still get the described error 
while
trying to "make check". I am attaching stderr and stdout from make check.

I have:

$ uname -m
i686

$ uname -r
2.6.24-19-generic

$ uname -s
Linux

$ python -V
Python 2.5.2

$ gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v
--enable-languages=c,c++,fortran,objc,obj-c++,treelang --prefix=/usr 
--enable-shared
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.2
--program-suffix=-4.2 --enable-clocale=gnu --enable-libstdcxx-debug 
--enable-objc-gc
--enable-mpfr --enable-targets=all --enable-checking=release 
--build=i486-linux-gnu
--host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.2.3 (Ubuntu 4.2.3-2ubuntu7)

Original comment by renato...@gmail.com on 28 Jul 2008 at 5:43

Attachments:

• make_check_stderr.txt
• make_check_stdout.txt

[GoogleCodeExporter]

I can confirm the bug on my Ubuntu 8.04 in v1.0.1. The problem is in handling
-gtest-output flag.

When -gtest-output=xml:out_name is specified, gtest_xml_outfiles_test.py 
expects the
binary (gtest_xml_outfile1_test_ or gtest_xml_outfile2_test_) to create file
gtest_xml_outfile1_test_.xml in the directory named out_name. Instead, the 
binary
tries to create the file named out_name (in the current directory if out_name is
relative path) and put the output there.

This is disconnect between the binary and the testing script and it is in 
1.0.1. I am
reopening the bug. Please mark it fixed again if it is fixed post 1.0.1.

Original comment by vladlosev on 9 Aug 2008 at 1:50

• Changed state: New

[GoogleCodeExporter]

I still cannot repro it on my Ubunto machine. :-(

The error log from renatoppl says:

AssertionError:
'/home/renatoppl/Downloads/gtest-1.0.1/test/gtest_xml_output_unittest_
--gtest_output=xml:/tmp/tmpmOmdlc/gtest_xml_output_unittest_out.xml &> 
/dev/null'
exited with code 0, which doesn't match the expected exit code 1.

gtest_xml_output_unittest.py does this:

    status = os.system(command)
    signal = status & 0xff
    self.assertEquals(0, signal,
                      "%s was killed by signal %d" % (gtest_prog_name, signal))
    exit_code = status >> 8
    self.assertEquals(expected_exit_code, exit_code,
                      "'%s' exited with code %s, which doesn't match "
                      "the expected exit code %s."
                      % (command, exit_code, expected_exit_code))

This matches the documentation for os.system() and works fine on my Ubuntu 
machine.

Vlad, since you can repro the bug on your machine, can you investigate it?

I don't think it has to do with the handling of -gtest_output.  I double 
checked the
code and found no mismatch between gtest_xml_outfiles_test.py and
gtest_xml_outfile1_test_.  Depending on whether out_name ends with "/", 
out_name is
treated either as the output file name or the directory that contains the output
file.  In either case, both the .py and the .cc treat it consistently.

Original comment by shiq...@gmail.com on 17 Sep 2008 at 9:43

• Changed state: Accepted

[GoogleCodeExporter]

The exit code problem *should* be fixed in the trunk now.  The fix will be in 
release
1.1.0.

I still cannot repro the xml-output-file-not-found problem.

Original comment by shiq...@gmail.com on 18 Sep 2008 at 6:27

[GoogleCodeExporter]

Sorry - I got ahead of myself.  The fix mentioned in the previous comment 
hasn't been
checked in to the trunk yet.  It is still being reviewed, and should be checked 
in soon.

Original comment by shiq...@gmail.com on 18 Sep 2008 at 6:34

[GoogleCodeExporter]

User Pete Johns reported on Aug 9, 2008:

Thanks Vlad.

I took a look at your comment on the bug <http://code.google.com/p/
googletest/issues/detail?id=9&q=xml#c4>, but you do not seem to be
describing the problem that I am experiencing. The file
(output_file_name2) *is* written in the correct directory
(output_dir_), but it is not ready for reading when the assertion is
called.

The following patch highlights the problem by waiting (and printing
dots) until the file appears.

If I get a chance to fix this over the weekend, I shall submit a
patch. I can confirm that the problem is the same on both Ubuntu and
Solaris.

Best;

--paj

Index: gtest_xml_outfiles_test.py
===================================================================
--- gtest_xml_outfiles_test.py  (revision 78)
+++ gtest_xml_outfiles_test.py  (working copy)
@@ -114,6 +114,8 @@
    output_file1 = os.path.join(self.output_dir_, output_file_name1)
    output_file_name2 = 'lt-' + output_file_name1
    output_file2 = os.path.join(self.output_dir_, output_file_name2)
+    while not (os.path.isfile(output_file1) or
os.path.isfile(output_file2)):
+        print '.',
    self.assert_(os.path.isfile(output_file1) or
os.path.isfile(output_file2),
                 output_file1)

Original comment by shiq...@gmail.com on 5 Nov 2008 at 9:13

[GoogleCodeExporter]

Vlad reported that he can repro the bug on his machine at work.

Original comment by shiq...@gmail.com on 5 Nov 2008 at 9:14

[GoogleCodeExporter]

I can confirm the problem still present in the latest version of Google Test on 
my
Ubuntu Hardy box with python 2.5.2, although it doesn't show up on a Dapper box 
with
python 2.4.3. Three tests fail:
test/gtest_break_on_failure_unittest.py
test/gtest_xml_output_unittest.py
test/gtest_xml_outfiles_test.py

In gtest_break_on_failure_unittest.py, the Run() function appears to always 
return
true. I can confirm that on Hardy os.WIFSIGNALED() never returns True in this 
test.
It is ither broken or used incorrectly.

In the other two tests, the files expected from the child process do not exist 
when
the script checks for their existence or tries to open them. But it is there 
when I
look it up later. Apparently the child process is still not done when 
os.system()
returns.  Has anyone else encountered it?

Original comment by vladlosev on 5 Nov 2008 at 11:58

[GoogleCodeExporter]

Issue 52 has been merged into this issue.

Original comment by vladlosev on 22 Nov 2008 at 5:46

[GoogleCodeExporter]

Some news on this issue. I have patches for two of the three failures. The 
reason
tuned out to be the use of "2>&1 &>/dev/null". On most Ubuntu systems, /bin/sh 
is
symlinked to /bin/bash. On Hardy, /bin/sh is symlinked to /bin/dash. Dash 
doesn't
understand &> construct (which is a bashism) and iterprets & separately, 
considering
it an instruction to execute program in the background. As a result, the python
script receives control and tries to inspect files before the test binary 
executes
ong enough to create them. The fixes are in revision 135.

The third failure is still not solved, so I'm leaving the bug open.

Original comment by vladlosev on 22 Nov 2008 at 5:57

[GoogleCodeExporter]

The issue finally fixed in release 1.2.0. The third problem was due to the way 
dash
behaved when python executed OS.system(). It was spawning an extra child 
process to
run the command which resulted in the signal termination status code being lost.

Original comment by vladlosev on 27 Nov 2008 at 6:15

• Changed state: Fixed