api-sanity-checker libsample_cpp test fails on Mac OS X 10.8 with Macports gcc 4.7

[cdeil]

I'd like to use the api-sanity-checker on Mac, but already for the C++ self-test I get this error:

$ cd libsample_cpp
$ make
gcc -dynamiclib -fkeep-inline-functions -x c++ libsample.cpp -o libsample.dylib
Undefined symbols for architecture x86_64:
  "vtable for __cxxabiv1::__class_type_info", referenced from:
      typeinfo for TestNS::throw_class in ccPqyNkM.o
      typeinfo for TestNS::A in ccPqyNkM.o
      typeinfo for TestNS::abstract_class in ccPqyNkM.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "vtable for __cxxabiv1::__si_class_type_info", referenced from:
      typeinfo for TestNS::C in ccPqyNkM.o
      typeinfo for TestNS::B in ccPqyNkM.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "___cxa_pure_virtual", referenced from:
      vtable for TestNS::throw_class in ccPqyNkM.o
      vtable for TestNS::abstract_class in ccPqyNkM.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make: *** [all] Error 1

I am using the Macports gcc 4.7:

$ which gcc
/opt/local/bin/gcc
$ gcc --version
gcc (MacPorts gcc47 4.7.2_2) 4.7.2
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[aponomarenko]

The tool is known to work on Mac OS X Leopard. But the best performance is on Linux.

Try to use gcc from Xcode instead of Macports gcc. Is this an option for you?

[cdeil]

The problem is that in the Makefile you generate you have gcc and not g++. Using g++ instead the C++ test works on Mac both with the Macports compiler or the System Apple compiler in /usr/bin/g++.

[pkgdiff]

The problem has been reproduced. The test is broken, but the tool works well. So, you can use it.

Please apply this patch if you want to fix the test:

--- api-sanity-checker.pl       2012-10-09 17:45:23.000000000 +0400
+++ api-sanity-checker.pl       2012-11-07 11:33:04.110348195 +0400
@@ -19401,8 +19401,14 @@
             $BuildCmd .= " -fPIC";
         }
     }
-    elsif($OSgroup eq "macos") {
-        $BuildCmd = "$GCC_PATH -dynamiclib -fkeep-inline-functions".($Lang eq "C++"?" -x c++":"")." libsample.$Ext -o libsample.$LIB_EXT";
+    elsif($OSgroup eq "macos")
+    {
+        if($Lang eq "C++") {
+            $BuildCmd = "$GCC_PATH -dynamiclib -lstdc++ -fkeep-inline-functions -x c++ libsample.$Ext -o libsample.$LIB_EXT";
+        }
+        else {
+            $BuildCmd = "$GCC_PATH -dynamiclib -fkeep-inline-functions libsample.$Ext -o libsample.$LIB_EXT";
+        }
     }
     else {
         $BuildCmd = "$GCC_PATH -shared -fkeep-inline-functions".($Lang eq "C++"?" -x c++":"")." libsample.$Ext -o libsample.$LIB_EXT";

[cdeil]

After applying your patch I now get this error for the test:

$ ./api-sanity-checker.pl --test

testing C library API
library(ies) analysis: [100.00%]
header(s) analysis: [25.00%]
Using C99 compatibility mode
header(s) analysis: [100.00%]
ERROR: cannot generate tests because extracted list of symbols is empty
result: FAILED (0 test cases, 0 passed, 0 failed)

testing C++ library API
library(ies) analysis: [100.00%]
header(s) analysis: [100.00%]
ERROR: cannot generate tests because extracted list of symbols is empty
result: FAILED (0 test cases, 0 passed, 0 failed)

$ cat logs/libsample_c/1.0/log.txt 
Temporary header file '/var/folders/sb/4qv5j4m90pz1rw7m70rj1b1r0000gn/T/gD0bPjrnfs/dump.h' with the following content will be compiled to create GCC syntax tree:

  // add includes
  #include "/Users/deil/code/api-sanity-checker/libsample_c/libsample.h"

The GCC parameters:
  gcc -fdump-translation-unit -fkeep-inline-functions -c -fpreprocessed -x objective-c++-header /var/folders/sb/4qv5j4m90pz1rw7m70rj1b1r0000gn/T/gD0bPjrnfs/dump.i

$ cat logs/libsample_cpp/1.0/log.txt 
Temporary header file '/var/folders/sb/4qv5j4m90pz1rw7m70rj1b1r0000gn/T/_81DBxyShS/dump.h' with the following content will be compiled to create GCC syntax tree:

  // add includes
  #include "/Users/deil/code/api-sanity-checker/libsample_cpp/libsample.h"

The GCC parameters:
  gcc -fdump-translation-unit -fkeep-inline-functions -c -x objective-c++-header /var/folders/sb/4qv5j4m90pz1rw7m70rj1b1r0000gn/T/_81DBxyShS/dump.h

[cdeil]

On my actual library I also get this error:

$ ./api-sanity-checker.pl -lib libgamma -d VERSION.xml -gen
library(ies) analysis: [100.00%]
header(s) analysis: [100.00%]
ERROR: cannot generate tests because extracted list of symbols is empty

Here's logs/libgamma/0.0/log.txt: https://gist.github.com/4030290

[aponomarenko]

What is the version of your Mac OS?

Could you please run this command on your dynamic library:

otool -TV B.dylib

Does it print a list of symbols?

Thanks.

[cdeil]

I'm on Mac OS X 10.8 with this Xcode:

$ xcodebuild -version
Xcode 4.5.2
Build version 4G2008a

otool -TV does not print a list of symbols:

$ otool -TV /usr/local/gamma/lib/libgamma.dylib 
/usr/local/gamma/lib/libgamma.dylib:
Table of contents (0 entries)
module name      symbol name

nm does print a list of symbols.

[aponomarenko]

Could you please also check if this command prints anything

otool -L libgamma.dylib

[cdeil]

Yes it does:

$ otool -L /usr/local/gamma/lib/libgamma.dylib 
/usr/local/gamma/lib/libgamma.dylib:
    /usr/local/gamma/lib/libgamma.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/gamma/lib/libreadline.6.2.dylib (compatibility version 6.0.0, current version 6.2.0)
    /usr/local/gamma/lib/libncurses.5.dylib (compatibility version 5.0.0, current version 5.0.0)
    libcfitsio.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

[aponomarenko]

Try to apply this patch to fix the problem:

--- api-sanity-checker.pl       2012-10-02 12:55:12.000000000 +0400
+++ api-sanity-checker.pl       2012-11-07 13:46:57.698404269 +0400
@@ -3607,16 +3607,16 @@
     $GLIBC_TESTING = 1 if($Lib_SoName=~/\Alibc.$LIB_EXT/ and not $IsNeededLib);
     if($OSgroup eq "macos")
     {
-        my $OtoolCmd = get_CmdPath("otool");
-        if(not $OtoolCmd)
+        my $NM = get_CmdPath("nm");
+        if(not $NM)
         {
-            print STDERR "ERROR: can't find \"otool\"\n";
+            print STDERR "ERROR: can't find \"nm\"\n";
             exit(1);
         }
-        open(DYLIB, "$OtoolCmd -TV $Lib_Path 2>$TMP_DIR/null |");
+        open(DYLIB, "$NM -jg $Lib_Path 2>$TMP_DIR/null |");
         while()
         {
-            if(/[^_]+\s+_([\w\$]+)\s*\Z/)
+            if(/\A_([\w\$]+)\s*\Z/)
             {
                 my $fullname = $1;
                 my ($realname, $version) = get_symbol_name_version($fullname);
@@ -3660,6 +3660,14 @@
             }
         }
         close(DYLIB);
+        
+        my $OtoolCmd = get_CmdPath("otool");
+        if(not $OtoolCmd)
+        {
+            print STDERR "ERROR: can't find \"otool\"\n";
+            exit(1);
+        }
+        
         open(DYLIB, "$OtoolCmd -L $Lib_Path 2>$TMP_DIR/null |");
         while()
         {

All the patches will go to 1.13

[cdeil]

With your last fix api-sanity-checker runs fine on Mac OS X 10.8 both for the test and my actual library.

Thank you very much!

Maybe for now you can push it to github:master until the new release it out? (for some reason your second patch didn't apply cleanly, so I made all the edits by hand, ... git pull is easier.)

[cdeil]

Actually I think I spoke too soon. There's still a problem with running the tests for my actual library.

None of the tests actually run because of a shared library loading problem, they all show up as api-sanity-checker warnings:

received signal TRAP

dyld: Library not loaded: /usr/local/gamma/lib/libreadline.6.2.dylib
  Referenced from: /usr/local/gamma/lib/libgamma.0.dylib
  Reason: image not found

I'll try to find out what the problem is.

Also I noticed that on Mac tests are generated for these functions which should not be part of the gammalib API:

stdio.h, libctools.0.dylib
[+] tmpnam ( char* p1 ) 
cmath, libgamma.0.dylib
[+] atan2 ( __float80 __y, __float80 __x ) 
[+] floor ( __float80 __x ) 
[+] fmod ( __float80 __x, __float80 __y ) 
[+] tan ( __float80 __x ) 
ctype.h, libgamma.0.dylib
[+] __tolower ( __darwin_ct_rune_t p1 ) 
[+] __toupper ( __darwin_ct_rune_t p1 ) 
...
new, libgamma.0.dylib
[+] operator delete ( void* p1 ) 
[+] operator delete[] ( void* p1 ) 
[+] operator new ( std::size_t p1 ) 
[+] operator new[] ( std::size_t p1 ) 
stdio.h, libgamma.0.dylib
[+] remove ( char const* p1 ) 
[+] sprintf ( char* p1, char const* p2, ... ) 
time.h, libgamma.0.dylib
[+] difftime ( time_t p1, time_t p2 ) 
[+] gmtime ( time_t const* p1 ) 
[+] time ( time_t* p1 ) 

It seems that some of the symbols (like tmpnam) are not in the shared library at all, others like atan2 or floor are undefined symbols:

$ nm /usr/local/gamma/lib/libgamma.dylib  | egrep 'tmpnam|atan2|floor'
0000000000086920 T __ZNK7GWcslib6atan2dERKdS1_
                 U _atan2
                 U _floor

I wonder if nm -jg is the correct way for the api-sanity-checker to get a list of symbols?

[cdeil]

The problem with the shared library loading has nothing to do with the api-sanity-checker or gammalib. I simply had removed /usr/local/gamma/lib/libreadline.6.2.dylib and had to re-install gammalib.

[aponomarenko]

Great! Latest changes have been pushed to repository.

Note, that you can add any gcc options to generated Makefiles by this option in the XML descriptor:

<gcc_options>
    ...
</gcc_options>

[aponomarenko]

The problem with tests for undefined symbols has been fixed in e097459839b5737730cb29bd86f60987fd287311

[cdeil]

Why did the api-sanity-checker generate this test? http://upstream-tracker.org/tests/gammalib/00.06.02/groups/new/functions/_ZdlPvS_/view.html I wouldn't have expected that since it's not (at least not intentionally) part of the gammalib API.

[cdeil]

The symbols on Mac are now similar to Linux.

On Mac I see these five classes in logs/libgamma/0.0/log.txt that I have no idea where they come from:

// add classes
...
  _category_t* tmp_add_class_176;
  _class_ro_t* tmp_add_class_177;
  _class_t* tmp_add_class_178;
  _message_ref_t* tmp_add_class_179;
  _protocol_t* tmp_add_class_180;
  _super_message_ref_t* tmp_add_class_181;

On Linux there's only one class in logs/libgamma/0.0/log.txt that I think shouldn't be there:

// add classes
...
  timex* tmp_add_class_176;

Here's the results I get on Mac for gammalib git devel, not 0.6.2: https://dl.dropbox.com/u/4923986/bug_reports/api-sanity-checker/mac_view_tests.html https://dl.dropbox.com/u/4923986/bug_reports/api-sanity-checker/mac_test_results.html

[aponomarenko]

The symbol _ZdlPvS_ is Weak on Linux and I think it's a bug that the tool generates a test case for it.

Could you please tell me the output of this command on your Mac OS?

nm -g libgamma.dylib | grep _ZdlPvS_

I think there may be a problem with distinguishing of this symbol from other "public" library symbols on Mac.

[cdeil]

Nothing found on Mac:

$ nm -g /usr/local/gamma/lib/libgamma.dylib | grep _ZdlPvS_
$

Here's the full output of nm -g /usr/local/gamma/lib/libgamma.dylib: https://gist.github.com/4032086

[aponomarenko]

Why did the api-sanity-checker generate this test? http://upstream-tracker.org/tests/gammalib/00.06.02/groups/new/functions/_ZdlPvS_/view.html I wouldn't have expected that since it's not (at least not intentionally) part of the gammalib API.

The generation of tests for WEAK symbols on Linux has been disabled in 1f0d4f8237ffcdf88241ee8a09461d1d9577fcad

The test suite for gammalib has been reloaded to Upstream Tracker.

[lvc]

Fixed in 1.98