Error when compiling to shared library

[davekch]

I have installed your library in /usr/local/lib. Compiling programs using it to executables works just fine. Now I have some code in example.cpp that relies on #include"cnpy.h" and I want to compile it to a shared object file example.so. So I tried

$ g++ -c -fPIC example.cpp --std=c++11
$ g++ example.o -shared -o libexample.so -no-pie -Wl,--whole-archive /usr/local/lib/libcnpy.a -Wl,--no-whole-archive

which results in this error:

/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
/usr/local/lib/libcnpy.a(cnpy.cpp.o): In function `load_the_npz_array(_IO_FILE*, unsigned int, unsigned int)':
cnpy.cpp:(.text+0x1535): undefined reference to `inflateInit2_'
cnpy.cpp:(.text+0x1597): undefined reference to `inflate'
cnpy.cpp:(.text+0x15ac): undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status

What am I doing wrong? Any help is appreciated. (I use Ubuntu 16.04 if that's important)

[rogersce]

Can you try adding -lz to your compile command?

[davekch]

adding it at the very end of the command:

g++ example.o -shared -o libexample.so -no-pie -Wl,--whole-archive /usr/local/lib/libcnpy.a -Wl,--no-whole-archive -lz

results in this error:

/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status

If I add -lz before specifying -Wl,...libcnpy.a the same error as above occurs.

[davekch]

I can compile without errors using the command g++ example.o -shared -o libexample.so -lcnpy -lz but I have a feeling that does not do what I want... Using libexample.so does not work as expected. Any further ideas about what's going on here? I must admit that I still don't quite understand what all the flags are doing and what effect they have as g++ example.o -shared -o libexample.so doesn't throw any errors either even if this obviously won't work because it doesn't include your library

[davekch]

I also figured that the errors of your suggested compilation command go away if I add a line int main(){return 0;} to example.cpp but then again example.so is not a shared library anymore. I don't understand.... Do you think this is a problem with your library or should I rather open a threat at stackexchange?

[rogersce]

Are you using a separate compile command and linking command? Add the -lz to the linking phase.

[rogersce]

So, if I understand correctly, you took the main function out of example.cpp and put something else in there that you want to compile into a shared library, that depends on cnpy? Did you also write a header file to facilitate using this shared library in other pieces of your code?

[rogersce]

Here's a simple example accomplishing what I think you're going for. Let me know if it works for what you're trying to do.

mylibrary.hpp

#pragma once
#include<cstddef>

bool do_something(size_t,size_t);

mylibrary.cpp

#include"cnpy.h"
#include<vector>

bool do_something(size_t x, size_t y)
{
    std::vector<double> data(x*y);
    for(size_t n = 0; n < data.size(); n++) data[n] = n;

    cnpy::npy_save("asdf.npy",data.data(),{x,y});
}

test.cpp

#include<iostream>
#include"mylibrary.hpp"

int main()
{
    std::cout<<"hello!\n";
    do_something(5,2);
}

The following 2 commands build the library and compile a test program to use the library. Note: my cnpy library is installed at $HOME. You may need to change paths to /usr/local

g++ -L$HOME/lib -I$HOME/include -O3 -shared -fPIC -o libmylibrary.so mylibrary.cpp -lcnpy

g++ -L$HOME/lib -L. -O3 -o test test.cpp -lmylibrary

[davekch]

So, if I understand correctly, you took the main function out of example.cpp and put something else in there that you want to compile into a shared library, that depends on cnpy?

That's right. My situation is the following: I have a python script that I want to speed up by extending it with fast C++ code, in the way suggested in this answer. This requires compiling the needed C++ code to a shared library.

grid.cpp

#include<vector>
#include"cnpy.h"
using namespace std;

class grid{
    private: 
        vector<vector<int>> matrix;
    public: 
        void calculate(){ .... }
        void save(){
            cnpy::save( "grid.npy", &matrix[0][0], {matrix.size(),matrix[0].size()}, "w"  );
        }
};

extern "C" {
    grid* grid_new(){ return new grid(); }
    void grid_calculate(grid* field){ field->calculate(); }
    void grid_save(grid* field){ field->save(); }
}

So far the following compilation commands don't produce errors:

$ g++ -c -fPIC grid.cpp --std=c++11
$ g++ grid.o -shared -o libgrid.so -lcnpy -lz 

and as well the one you just suggested:

g++ -L/usr/local/lib -I/usr/local/include -O3 -shared -fPIC -o libgrid.so grid.cpp -lcnpy --std=c++11

But if I try to use the produced .so file in a python script (like in the stackoverflow answer I linked above):

test.py

from ctypes import cdll
grid = cdll.LoadLibrary("./libgrid.so")

class grid(object):
    def __init__(self):
        self.obj = grid.grid_new()
    # ...

field = grid()

I get the error

Traceback (most recent call last):
  File "test.py", line 30, in <module>
    field = grid()
  File "test.py", line 18, in __init__
    self.obj = grid.grid_new()
AttributeError: type object 'grid' has no attribute 'grid_new'

I don't even know how to troubleshoot this but my guess would be that the linking issues are now resolved and that something's wrong with my code...

[rogersce]

in your python code, it looks like class grid overwrites the grid dll in your global namespace. try this instead

from ctypes import cdll
grid = cdll.LoadLibrary("./libgrid.so")

class Grid(object):
    def __init__(self):
        self.obj = grid.grid_new()
    # ...

field = Grid()

[davekch]

You're totally right, that was just stupid of me. Thank you so much for your help! Everything works fine now.

[rogersce]

👍 glad it works, I'm going to close, in that case.