XML support is broken in v4

[galbramc]

I'm getting a segfault when passing any argument (besides -h) to a unit test that I have compiled.

If I run the unit test everything works fine:

./cooltest 
.           3

  e3_Tt3  0.13432114E-03       e3_Tt3  0.66721437E-04       e3_Tt3  0.33991869E-04   
          0.13432114E-03               0.66721437E-04               0.33991869E-04   

  e3_Tt4  0.67160496E-04       e3_Tt4  0.62384470E-04       e3_Tt4  0.55132664E-04   
          0.67160496E-04               0.62384470E-04               0.55132664E-04   

  e3_Trr   0.0000000           e3_Trr  0.14678698           e3_Trr  0.25944791       
           0.0000000                   0.14678698                   0.25944791       

Time:         0.001 seconds

 OK
 (1 test)

but with an argument

./cooltest -v

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x7fa684707720 in ???
#1  0x7fa684706945 in ???
#2  0x7fa68413dcaf in ???
#3  0x4c1ec2 in __fp_argparser_MOD_parse_args_kludge_args
    at /home/galbramc/util/pFUnit/extern/fArgParse/src/ArgParser.F90:331
#4  0x4c2b94 in __fp_argparser_MOD_parse_args_kludge_command_line
    at /home/galbramc/util/pFUnit/extern/fArgParse/src/ArgParser.F90:279
#5  0x40b1d0 in __funit_MOD_run
    at /home/galbramc/util/pFUnit/src/funit/FUnit.F90:69
#6  0x416f8f in funit_main_
    at /home/galbramc/util/pFUnit/src/funit/funit_main.F90:16
#7  0x403265 in MAIN__
    at /home/galbramc/util/pFUnit/build/installed/PFUNIT-4.1/include/driver.F90:47
#8  0x4032b1 in main
    at /home/galbramc/util/pFUnit/build/installed/PFUNIT-4.1/include/driver.F90:19
Segmentation fault (core dumped)

I'm using gfortran-8 on Ubuntu 14.04 with the master branch. Any help would me much appreciated, I can try to provide more information if needed.

[tclune]

Sorry -somehow I missed several GitHub emails over the last week and only now saw your ticket when looking at a different issue.

Can you tell me the specific version of gfortran (8.2 vs 8.3)? I will then try to reproduce. I've not used the "-v" option recently, but as the stack trace shows (_argskludge) I've had to implement compiler specific workarounds for compiler bugs in that section of the code.

[galbramc]

That's annoying that you have to deal with compiler bugs. I am actually trying to use the --xml argument, but I noticed none of the arguments worked.

Here are the compilers I would like to use:

gfortran-8 --version
GNU Fortran (Ubuntu 8.3.0-16ubuntu3~16.04) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

gfortran-9 --version
GNU Fortran (Ubuntu 9.1.0-2ubuntu2~16.04) 9.1.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 ifort --version
ifort (IFORT) 19.0.4.243 20190416
Copyright (C) 1985-2019 Intel Corporation.  All rights reserved.

[tclune]

OK, there are actually several issues here. The good news is that I do not think this is a compiler issue per se. First, there is no "-v" option. There are "-d", "--debug", and "--verbose". I can fix this, but it is adhering to the python argparse conventions here. (The fix is to split this into 2 options and make it verbose if either debug or verbose are set.)

Of course, there should have been a much more informative message about those options not existing. I've not checked yet whether that is a compiler bug or a code bug on my part. Most likely the latter as NAG Fortran is also crashing in a similar manner when I use an unsupported flag.

Then, there is shockingly no --xml option. I know I glossed over the XML printer in the major refactoring for version 4, but the XML tests still were passing. So I failed to notice that it is no longer properly integrated.

I've create a local branch and have made a few changes and I can now get xml output. But before merge this into development I want to think a bit more about the relationship between "printers" and "test listeners". Mostly this actually means I want to go stare at JUnit a bit more and see if I'm missing something. E.g., my current fix allows one to either use the XML printer or the default printer. But it really should be more like an option of doing both. (I can provide more details if you really care.)

I'm going to split this issue into 2 separate issues. This one will be about getting XML working. The other will be about ensuring users get an informative message when using an unsupported command line option.

[galbramc]

So I was looking at section "4.3.2 Command Line Options" in the pFUnit3-ReferenceManual.pdf assuming that things had not changed too much from version 3 to 4. There is a "-v" option listed there, and hence why I tried it out.

I would very much appreciate if the JUnit xml outputs could be revived. I am trying to use pFUnit in conjunction with Jenkins, and the xml output was one of the reasons why I chose to use pFUnit as opposed some of the other frameworks out there.

In general I do think pFUnit is very well thought out, and I appreciate all the work you have put into creating this.

[tclune]

@galbramc

Thanks for the encouragement. I ran into some unfortunate issues with travis today which consumed far more time than expected. I just pushed the fix for the fArgParse bug, and have the pFUnit change essentially ready to go. Should be able to merge and release in the morning.

[galbramc]

So I still can't specify an xml file with this change, and I still get a segfault when I provide a command that is not available.

cooltest --xml test.xml

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x10dd1839d
#1  0x10dd1782b
#2  0x7fff6761eb5c
#3  0x10dc2994c
#4  0x10dc2a65f
#5  0x10db51b37
#6  0x10db5ce97
#7  0x10db499cd
#8  0x10db49a1b
Segmentation fault: 11

[tclune]

Did you do a fresh clone of pFUnit? I'm wondering if somehow you have a stale fArgParse submodule. I get the output below from all 3 of my local compilers (ifort 19.0.5, gfortran 9.2.0_1, and nag 6.2). I can probably get access to gfortran 9.1 on our cluster, but would appreciate it if you could double check your latest installation first.

./tests/funit-core/new_tests.x --xml
<testsuite name="" errors="0" failures="0" tests="204" time=".0050">
<testcase name="AssertString_suite.testAssertEqualStringSame"/>
<testcase name="AssertString_suite.testAssertEqualStringDifferent"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnAllWhite1"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnAllWhite2"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnAllWhite3"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnAllWhite4"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnAllWhite5"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnAllWhite6"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnAllWhite7"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif1"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif2"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif3"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif4"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif5"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif6"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif7"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif8"/>
<testcase name="AssertString_suite.testAssertEqualStrIgnWhiDif9"/>
<testcase name="AssertString_suite.testAssertEqualStringTrimWhitespace1"/>
<testcase name="AssertString_suite.testAssertEqualStringTrimWhitespace2"/>
<testcase name="AssertString_suite.testAssertEqualStringTrimWhitespace3"/>
<testcase name="AssertString_suite.testAssertEqualStringTrimWhitespace4"/>
<testcase name="AssertString_suite.testAssertEqualStringKeepWhitespace1"/>
<testcase name="AssertString_suite.testAssertEqualStringKeepWhitespace2"/>
<testcase name="AssertString_suite.testAssertEqualStringKeepWhitespace3"/>
<testcase name="AssertString_suite.testAssertEqualStringKeepWhitespace4"/>
<testcase name="AssertString_suite.testAssertEqualNonzeroBlanks1"/>
<testcase name="AssertString_suite.testAssertEqualNonzeroBlanks2"/>
<testcase name="AssertString_suite.testAssertEqualNonzeroBlanks3"/>
<testcase name="AssertString_suite.testAssertEqualNonzeroBlanks4"/>
<testcase name="AssertInteger_suite.test_AssertAssociated_0d"/>
<testcase name="AssertInteger_suite.testAssertEqual_equal"/>
<testcase name="AssertInteger_suite.testAssertEqual_unequal"/>
<testcase name="AssertInteger_suite.testAssertEqual_unequalWithMessage"/>
<testcase name="AssertInteger_suite.testAssertEqual1D1D_equal"/>
<testcase name="AssertInteger_suite.testAssertEqual1D1D_nonconforming"/>
<testcase name="AssertInteger_suite.testAssertEqual1D1D_conforming"/>
<testcase name="AssertInteger_suite.testAssertEqual1D1D_unequalA"/>
<testcase name="AssertInteger_suite.testAssertEqual1D1D_unequalB"/>
<testcase name="AssertInteger_suite.testAssertEqual2D2D_equal"/>
<testcase name="AssertInteger_suite.testAssertEqual2D2D_nonconforming"/>
<testcase name="AssertInteger_suite.testAssertEqual2D2D_unequal"/>
<testcase name="AssertInteger_suite.testAssertLessThan_falseA"/>
<testcase name="AssertInteger_suite.testAssertLessThan_falseB"/>
<testcase name="AssertInteger_suite.testAssertLessThan_true"/>
<testcase name="AssertInteger_suite.testAssertLessThanOrEqual_false"/>
<testcase name="AssertInteger_suite.testAssertLessThanOrEqual_trueA"/>
<testcase name="AssertInteger_suite.testAssertLessThanOrEqual_trueB"/>
<testcase name="AssertInteger_suite.testAssertGreaterThan_falseA"/>
<testcase name="AssertInteger_suite.testAssertGreaterThan_falseB"/>
<testcase name="AssertInteger_suite.testAssertGreaterThan_true"/>
<testcase name="AssertInteger_suite.testAssertGreaterThanOrEqual_false"/>
<testcase name="AssertInteger_suite.testAssertGreaterThanOrEqual_trueA"/>
<testcase name="AssertInteger_suite.testAssertGreaterThanOrEqual_trueB"/>
<testcase name="AssertEqual_Real_suite.test_equal_scalar"/>
<testcase name="AssertEqual_Real_suite.test_equal_scalar_with_tolerance"/>
<testcase name="AssertEqual_Real_suite.test_equal_1D_actual"/>
<testcase name="AssertEqual_Real_suite.test_equal_1D_actual_with_tolerance"/>
<testcase name="AssertEqual_Real_suite.test_equal_2D_actual"/>
<testcase name="AssertEqual_Real_suite.test_equal_2D_actual_with_tolerance"/>
<testcase name="AssertEqual_Real_suite.test_infinity"/>
<testcase name="AssertEqual_Real_suite.test_infinity_1d"/>
<testcase name="AssertEqual_Real_suite.test_conformability_a"/>
<testcase name="AssertEqual_Real_suite.test_conformability_b"/>
<testcase name="AssertEqual_Real_suite.test_equal_fail_scalar_no_tolerance"/>
<testcase name="AssertEqual_Real_suite.test_equal_fail_scalar_with_tolerance"/>
<testcase name="AssertEqual_Real_suite.test_equal_fail_1d_with_tolerance"/>
<testcase name="AssertEqual_Real_suite.testEquals_2D_SingleElementDifferent"/>
<testcase name="AssertEqual_Real_suite.testEquals_2d_MultipleDiffs"/>
<testcase name="AssertEqual_Real_suite.testEquals_ScalarInf_unequal_A"/>
<testcase name="AssertEqual_Real_suite.testEquals_ScalarInf_unequal_B"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_scalar"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_scalar_with_tolerance"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_1D_actual"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_1D_actual_with_tolerance"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_2D_actual"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_2D_actual_with_tolerance"/>
<testcase name="AssertNotEqual_Real_suite.test_infinity"/>
<testcase name="AssertNotEqual_Real_suite.test_infinity_1d"/>
<testcase name="AssertNotEqual_Real_suite.test_conformability_a"/>
<testcase name="AssertNotEqual_Real_suite.test_conformability_b"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_fail_scalar_no_tolerance"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_fail_scalar_with_tolerance"/>
<testcase name="AssertNotEqual_Real_suite.test_notEqual_fail_1d_with_tolerance"/>
<testcase name="AssertNotEqual_Real_suite.testNotEqual_2D_SingleElementDifferent"/>
<testcase name="AssertNotEqual_Real_suite.testNotEqual_2d_MultipleDiffs"/>
<testcase name="AssertNotEqual_Real_suite.testNotEqual_ScalarInf_unequal_A"/>
<testcase name="AssertNotEqual_Real_suite.testNotEqual_ScalarInf_unequal_B"/>
<testcase name="AssertRelativelyEqual_Real_suite.test_relatively_equal_scalar"/>
<testcase name="AssertRelativelyEqual_Real_suite.test_relatively_equal_1D_actual"/>
<testcase name="AssertRelativelyEqual_Real_suite.test_rel_equal_fail_scalar_with_tolerance"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_scalar"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_scalar_with_tolerance"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_scalar_epsilon"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_1D_actual"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_1D_actual_with_tolerance"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_2D_actual"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_2D_actual_with_tolerance"/>
<testcase name="AssertLessThan_Real_suite.test_infinity"/>
<testcase name="AssertLessThan_Real_suite.test_conformability_a"/>
<testcase name="AssertLessThan_Real_suite.test_conformability_b"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_fail_scalar_no_tolerance"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_fail_scalar_with_tolerance"/>
<testcase name="AssertLessThan_Real_suite.test_lessThan_fail_1d_with_tolerance"/>
<testcase name="AssertLessThan_Real_suite.testLessThan_2D_SingleElementDifferent"/>
<testcase name="AssertLessThan_Real_suite.testLessThan_2d_MultipleDiffs"/>
<testcase name="AssertLessThan_Real_suite.testLessThan_ScalarInf_unlessThan_A"/>
<testcase name="AssertLessThan_Real_suite.testLessThan_ScalarInf_unlessThan_B"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_scalar"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_scalar_with_tolerance"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_scalar_epsilon"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_1D_actual"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_1D_actual_with_tolerance"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_2D_actual"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_2D_actual_with_tolerance"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_infinity"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_conformability_a"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_conformability_b"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_fail_scalar_no_tolerance"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_fail_scalar_with_tolerance"/>
<testcase name="AssertLessThanOrEqual_Real_suite.test_lessThan_fail_1d_with_tolerance"/>
<testcase name="AssertLessThanOrEqual_Real_suite.testLessThanOrEquals_2D_SingleElementDifferent"/>
<testcase name="AssertLessThanOrEqual_Real_suite.testLessThanOrEquals_2d_MultipleDiffs"/>
<testcase name="AssertLessThanOrEqual_Real_suite.testLessThanOrEquals_ScalarInf_unlessThan_A"/>
<testcase name="AssertLessThanOrEqual_Real_suite.testLessThanOrEquals_ScalarInf_unlessThan_B"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_scalar"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_scalar_with_tolerance"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_scalar_epsilon"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_1D_actual"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_1D_actual_with_tolerance"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_2D_actual"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_2D_actual_with_tolerance"/>
<testcase name="AssertGreaterThan_Real_suite.test_infinity"/>
<testcase name="AssertGreaterThan_Real_suite.test_conformability_a"/>
<testcase name="AssertGreaterThan_Real_suite.test_conformability_b"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_fail_scalar_no_tolerance"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_fail_scalar_with_tolerance"/>
<testcase name="AssertGreaterThan_Real_suite.test_lessThan_fail_1d_with_tolerance"/>
<testcase name="AssertGreaterThan_Real_suite.testGreaterThans_2D_SingleElementDifferent"/>
<testcase name="AssertGreaterThan_Real_suite.testGreaterThans_2d_MultipleDiffs"/>
<testcase name="AssertGreaterThan_Real_suite.testGreaterThans_ScalarInf_unlessThan_A"/>
<testcase name="AssertGreaterThan_Real_suite.testGreaterThans_ScalarInf_unlessThan_B"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_scalar"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_scalar_with_tolerance"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_scalar_epsilon"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_1D_actual"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_1D_actual_with_tolerance"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_2D_actual"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_2D_actual_with_tolerance"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_infinity"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_conformability_a"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_conformability_b"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_fail_scalar_no_tolerance"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_fail_scalar_with_tolerance"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.test_greaterThan_fail_1d_with_tolerance"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.testGreaterThanOrEquals_2D_SingleElementDiff"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.testGreaterThanOrEquals_2d_MultipleDiffs"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.testGreaterThanOrEquals_ScalarInf_ungreaterT"/>
<testcase name="AssertGreaterThanOrEqual_Real_suite.testGreaterThanOrEquals_ScalarInf_ungreaterT"/>
<testcase name="AssertEqual_Complex_suite.test_equal_scalar"/>
<testcase name="AssertEqual_Complex_suite.test_equal_scalar_with_tolerance"/>
<testcase name="AssertEqual_Complex_suite.test_equal_1D_actual"/>
<testcase name="AssertEqual_Complex_suite.test_equal_1D_actual_with_tolerance"/>
<testcase name="AssertEqual_Complex_suite.test_conformability_a"/>
<testcase name="AssertEqual_Complex_suite.test_conformability_b"/>
<testcase name="AssertEqual_Complex_suite.test_equal_fail_scalar_no_tolerance"/>
<testcase name="AssertEqual_Complex_suite.test_equal_fail_scalar_with_tolerance"/>
<testcase name="AssertEqual_Complex_suite.test_equal_fail_1d_with_tolerance"/>
<testcase name="AssertEqual_Complex_suite.testEquals_2D_SingleElementDifferent"/>
<testcase name="AssertEqual_Complex_suite.testEquals_2d_MultipleDiffs"/>
<testcase name="Norm_suite.Test_Linfinity_Norm_scalar"/>
<testcase name="Norm_suite.Test_Linfinity_Norm_1d"/>
<testcase name="Norm_suite.Test_L2Norm_scalar"/>
<testcase name="Norm_suite.Test_L2Norm_1d"/>
<testcase name="Norm_suite.Test_L1Norm_scalar"/>
<testcase name="Norm_suite.Test_L1Norm_1d"/>
<testcase name="Norm_suite.Test_Linfinity_Norm_scalar_complex"/>
<testcase name="Norm_suite.Test_Linfinity_Norm_1d_complex"/>
<testcase name="Norm_suite.Test_L2Norm_scalar_complex"/>
<testcase name="Norm_suite.Test_L2Norm_1d_complex"/>
<testcase name="Norm_suite.Test_L1Norm_scalar_complex"/>
<testcase name="Norm_suite.Test_L1Norm_1d_complex"/>
<testcase name="Disable_suite.test_disable"/>
<testcase name="TapListener_suite.test_addSuccess"/>
<testcase name="TapListener_suite.test_addFailure_nomessage"/>
<testcase name="TapListener_suite.test_addFailure_withmessage"/>
<testcase name="ExceptionList_suite.test_throw_message"/>
<testcase name="ExceptionList_suite.test_catch_fail"/>
<testcase name="GlobPattern_suite.test_patterns"/>
<testcase name="LiteralPattern_suite.test_match_found"/>
<testcase name="LiteralPattern_suite.test_match_not_found"/>
<testcase name="LiteralPattern_suite.test_match_various"/>
<testcase name="DotPattern_suite.test_match_found"/>
<testcase name="DotPattern_suite.test_match_not_found"/>
<testcase name="RepeatPattern_suite.test_match_zero"/>
<testcase name="RepeatPattern_suite.test_match_one"/>
<testcase name="RepeatPattern_suite.test_match_two"/>
<testcase name="RepeatPattern_suite.test_match_multichar"/>
<testcase name="NameFilter_suite.test_matches_exact"/>
<testcase name="NameFilter_suite.test_does_not_match_exact"/>
<testcase name="RegularExpression_suite.test_basic"/>
<testcase name="RegularExpression_suite.test_dot"/>
<testcase name="RegularExpression_suite.test_wrong_legth"/>
<testcase name="RegularExpression_suite.test_asterisk"/>
</testsuite>
bash-3.2$ 

[galbramc]

I blew away the "extern" directory in my build directory, and that seems to have fixed the xml

./cooltest --xml
           3

  e3_Tt3  0.13432114E-03       e3_Tt3  0.66721437E-04       e3_Tt3  0.33991869E-04   
          0.13432114E-03               0.66721437E-04               0.33991869E-04   

  e3_Tt4  0.67160496E-04       e3_Tt4  0.62384470E-04       e3_Tt4  0.55132664E-04   
          0.67160496E-04               0.62384470E-04               0.55132664E-04   

  e3_Trr   0.0000000           e3_Trr  0.14678698           e3_Trr  0.25944791       
           0.0000000                   0.14678698                   0.25944791       
<testsuite name="" errors="0" failures="0" tests="1" time=".0020">
<testcase name="cooltest_suite.coold"/>
</testsuite>

but I still see the segfault it I give it a bad argument

./cooltest --xm

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x106ea539d
#1  0x106ea482b
#2  0x7fff6761eb5c
#3  0x106db194c
#4  0x106db265f
#5  0x106cd9b37
#6  0x106ce4e97
#7  0x106cd19cd
#8  0x106cd1a1b
Segmentation fault: 11

[galbramc]

I'm also noticing that the xml output is only dumped to the screen. Would it be possible to have it duped to a file?

[tclune]

OK - I'll try a fresh clone myself. Maybe I'm the one that has not fully pushed to the updated version of fArgParse on the main repo.

You should be able to redirect the output to a different file with the "-o" ("--output") option:

[tclune]

fArgParse is current on the master, and I tested a fresh clone with all 3 of the compilers on my laptop.

It turns out I also still have gfortran 8.3 on my system (OSX) and it seems fine:

bash-3.2$ ./tests/funit-core/new_tests.x --xm
................................................................................
................................................................................
.......................I.....................
Time:         0.006 seconds

 OK
 (204 tests, 1 disabled)

[galbramc]

That is very strange... Let me try a fresh clone as well. Would it not be better to error out gracefully if someone does provide an unknown option? I prefer that over silently ignoring my mistake.

In the mean time, I didn't notice the -o option due to the misleading help string

  -f, --filter               only run tests that match pattern
  -o, --output               only run tests that match pattern

[galbramc]

Yeah the fresh clone does not segfault with a bad input. Some dependency must not be quite right in the cmake files... Very strange... CMake is supposed to be good at tracking these things so I don't have to blow everything away to update and recompile.

[galbramc]

One last thing, what report format is the xml files? I'm using "Publish xUnit test result report" in Jenkins, and there are a large number of report format to choose from. I've tried JUnit along with a few others, but none seem to be able to parse the xml files.

[tclune]

The XmlPrinter was contributed code. It is possible that some aspect was broken/dropped when I went to 4.0, but more likely there is metadata that was never properly introduced. It should be very straightforward to improve the facility if you have a specific report format in mind. I can also try to contact the original contributor to gauge their interest in an update. (Am guessing that user is still using 3.x or Xml would not have remained broken in 4.0 for so long.)

[tclune]

Depending on how you were doing the install, it is possible that find_package() was seeing the earlier install of fArgParse and thus not recompiling regardless of what you did in your build directory. I made that mistake at one point this afternoon in one of my shells.

Probably not so much a cmake bug (on their part or my part) but a subtle trap when external projects are installed in an environment where CMake can see them at the next iteration.

[galbramc]

I don't have a specific report format in mind. Which ever one that is easiest to implement that the Jenkins "Publish xUnit test result report" plugin can read would be perfectly fine.

[galbramc]

Just wanted to ping you to see if you made any progress on the xml reporting of if you needed some help from me with it?

[tclune]

I will need help at least in the form of specific changes to make. I don't use this feature myself and am unlikely to spend time reading the Jenkins documentation you mentioned given my current workload.

[galbramc]

Sorry for the delay in getting back to you. I know what you mean about being busy.

Here is the list of XML formats that the Jenkins xUnit plugin supports:

• JUnit (supported schema are Ant junit and Maven Surefire)
• AUnit
• MSTest (imported from MSTest Plugin)
• NUnit (imported from NUnit Plugin)
• UnitTest++
• Boost Test Library
• PHPUnit
• Free Pascal Unit
• CppUnit
• MbUnit
• Googletest
• EmbUnit
• gtester/glib
• QTestLib

If you have a preferred format I can help you dig up details on it.

It is also possible to create a custom XSL file that xUnit can use to parse the xml files already written by pFUnit. I'd have to read up on how an XSL file works to help create one though.