Why does -gen and -build fail for some tests?

[cdeil]

For my library a few % of the tests fail in the -gen as well as the -build phase when I run the api-sanity-checker.

Could you please add some information to the documentation on whether this is normal or if it indicates a problem with my library.

Thanks!

[cdeil]

Found this:

$ ls -1 tests/libgamma/0.0/
build_fail_list
gen_fail_list
groups
run_fail_list
scenario
view_tests.html

Here's how to get a list of failed builds:

$ head -n2 tests/libgamma/0.0/build_fail_list
tests/libgamma/0.0/groups/GSparseMatrix/functions/_Z9cs_cumsumPiS_i
tests/libgamma/0.0/groups/GEventCube/classes/GEventCube/_ZN10GEventCube12free_membersEv

And then to check why builds failed:

$ cat tests/libgamma/0.0/groups/GSparseMatrix/functions/_Z9cs_cumsumPiS_i/build_log 
test.cpp: In function 'int main(int, char**)':
test.cpp:11:24: error: 'cs_cumsum' was not declared in this scope
make: *** [test] Error 1

$ cat tests/libgamma/0.0/groups/GEventCube/classes/GEventCube/_ZN10GEventCube12free_membersEv/build_log 
test.cpp:146:12: error: invalid covariant return type for 'virtual GBase* GPointing_SubClass::clone() const'
In file included from /usr/local/gamma/include/gammalib/GObservation.hpp:35:0,
                 from /usr/local/gamma/include/gammalib/GObservations.hpp:34,
                 from /usr/local/gamma/include/gammalib/GammaLib.hpp:137,
                 from test.cpp:1:
/usr/local/gamma/include/gammalib/GPointing.hpp:56:28: error:   overriding 'virtual GPointing* GPointing::clone() const'
test.cpp:256:14: error: invalid covariant return type for 'virtual GEvents* GEventCube_SubClass::clone() const'
In file included from /usr/local/gamma/include/gammalib/GammaLib.hpp:142:0,
                 from test.cpp:1:
/usr/local/gamma/include/gammalib/GEventCube.hpp:58:25: error:   overriding 'virtual GEventCube* GEventCube::clone() const'
test.cpp:279:19: error: invalid covariant return type for 'virtual const GEvent* GEventCube_SubClass::operator[](const int&) const'
In file included from /usr/local/gamma/include/gammalib/GammaLib.hpp:142:0,
                 from test.cpp:1:
/usr/local/gamma/include/gammalib/GEventCube.hpp:54:30: error:   overriding 'virtual const GEventBin* GEventCube::operator[](const int&) const'
make: *** [test] Error 1

Is this normal that the api-sanity-checker generates some test cases for C++ libraries that don't compile?

There also is this list of gen failures, but as far as I can see there is no way to check why the generation failed?

$ head -n2 tests/libgamma/0.0/gen_fail_list 
_ZN10GEventAtom12copy_membersERKS_
_ZN10GEventCube12copy_membersERKS_
$ find . -name '_ZN10GEventAtom12copy_membersERKS_'
$

So is it normal that for some tests -gen fails and the user shouldn't care?

[aponomarenko]

Try to run the tool with -debug additional option. The file debug/libgamma/0.0/log.txt should be created with some explanations of why the tool cannot generate some tests.

[cdeil]

For gammalib I get this in the debug log:

Failed to initialize parameters of these symbols:
  GFitsImageLongLong::GFitsImageLongLong [in-charge] (int nx, long long const* pixels)
  GCTAModelRadialRegistry::copy_members (GCTAModelRadialRegistry const& registry)
  GEvents::GEvents [in-charge] (GEvents const& events)
...
Failed to create instances for these classes:
  GEvents
  GPointing
  GEventBin
  GResponse

Full log here: https://gist.github.com/4047621

Basically what I want to know is: is there something wrong with gammalib or is it this normal?

[aponomarenko]

The test generator has been fixed in 1404db74618de5bc61b36564a379133082d80232 and for now it can generate most of the tests.

Generally, if the generator cannot generate some test then this may indicate some inconsistency in the API. But in the case of gammalib all seems clear to me.

The only inconsistency I see is that generated Makefiles contain additional -lgomp option for linker. This indicates, that libgamma.so shared object doesn't require libgomp.so:

readelf -Wa libgamma.so|grep NEEDED

 0x00000001 (NEEDED)                     Shared library: [libstdc++.so.6]
 0x00000001 (NEEDED)                     Shared library: [libreadline.so.6]
 0x00000001 (NEEDED)                     Shared library: [libncurses.so.5]
 0x00000001 (NEEDED)                     Shared library: [libcfitsio.so.0]
 0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]

But libgamma.so requires some GOMP_* symbols:

readelf -Wa libgamma.so|grep GOMP_

   413: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_critical_end
  3562: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_sections_start
  4095: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_parallel_start
  4678: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_sections_end
  7367: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_critical_start
  7909: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_parallel_end
  9018: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_sections_next
  9399: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_barrier
  4697: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_critical_end
  7846: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_sections_start
  8379: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_parallel_start
  8962: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_sections_end
 11651: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_critical_start
 12193: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_parallel_end
 13302: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_sections_next
 13683: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND GOMP_barrier

[cdeil]

@aponomarenko Thanks for all your testing and improvements for gammalib. I just checked, the OpenMP linking issue is resolved in the git devel branch. We have the api-sanity-checker running in our continuous integration system for gammalib, let us know when you cut a new release and we'll update.

[aponomarenko]

Great!

I should release the new API Sanity Checker 1.98 next week. It will depend on the ABICC tool.