search

ibayer / fastFM

fastFM: A Library for Factorization Machines
http://ibayer.github.io/fastFM
Other
1.07k stars 206 forks source link

cython-wrapper: modified setup.py to link against ffm2. Added cpp_ffm… #94

Closed caos21 closed 7 years ago

caos21 commented 7 years ago

it compiles, but I did not check, I am thinking about how to do this. I got one warning but is not important.

caos21 commented 7 years ago

Ok. A possible solution that works to me is to package all the required libraries into one libfastFM

$ ar -M <<EOM                                                     
    CREATE libfastFM.a
    ADDLIB libfastFMd.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libgtest_maind.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libgmock_maind.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libglogd.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libprotobufd.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libzd.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libgtestd.a
    SAVE
    END
EOM

ranlib libfastFM.a

$ g++ little.cpp -std=c++11 -I/home/ben/.hunter/_Base/033a6ff/14d0f80/e1266bb/Install/include/eigen3/ -rdynamic fastFM/libfastFM.a -lpthread

$ ./a.out 

[ii] Little test

$ ldd fastFM/libfastFM.a 
        not a dynamic executable

$ ldd a.out 
        linux-vdso.so.1 =>  (0x00007ffdcb4d7000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8a0cf45000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8a0cbbd000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8a0c9a5000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8a0c5d5000)
        /lib64/ld-linux-x86-64.so.2 (0x000055afdd91f000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8a0c2c5000)
caos21 commented 7 years ago

Note also that in my system I always need to include eigen. Maybe you have a system wide installation of eigen.

ibayer commented 7 years ago

I use a different little.cpp. There is not need for Eigen types. https://gist.github.com/ibayer/7484fa620de10228ab4473453b1fe38f

ibayer commented 7 years ago

These are not needed

   ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libgtest_maind.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libgmock_maind.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libprotobufd.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libzd.a
    ADDLIB /home/ben/.hunter/_Base/033a6ff/14d0f80/71979ef/Install/lib/libgtestd.a
caos21 commented 7 years ago

is there a way to know the path used by hunter and then use it in setup.py? user variables

because if not it will be hardwired

caos21 commented 7 years ago

Sorry about the delay. I needed to correct some things:

  1. It's not OK to compile sources in a shared directory in virtualbox. Some relative paths got messed and try to read the host system.

  2. The old problem that cython/setup.py doesn't resolve paths outside the main path. Then we need fastFM2 as a subdirectory of fastFM

  3. In the virtualbox using pip needs --user flag because if not it tries to install in a system path. Before I didn't have this problem because in my host system I am using anaconda, I guess.

  4. We need the -fPIC flag everywhere, for fastFM2 is easily, added:

    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fPIC")

This is the problem to solve, we need Position Independent Code for all dependencies of fastFM2. The error when building the library is:

/usr/bin/ld: /home/ben/.hunter/_Base/033a6ff/14d0f80/e1266bb/Install/lib/libglog.a(logging.cc.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/home/ben/.hunter/_Base/033a6ff/14d0f80/e1266bb/Install/lib/libglog.a: error adding symbols: Bad value

This means that we need to tell hunter to recompile the libraries with -fPIC flag.

I think it is doable by creating a config.cmake with the info:

option(HUNTER_STATUS_DEBUG "Hunter debug info" ON)
hunter_config(glog VERSION 0.3.4-p1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(Eigen VERSION 3.3.1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(Protobuf VERSION 3.0.0-p1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(GTest VERSION 1.8.0-hunter-p5 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(gflags VERSION 2.1.2-p1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)

if I do not put glog dependency in setup.py everything runs well until you try to import in python:

$ python 
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ffm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: ./ffm.so: undefined symbol: blas_thread_shutdown_
>>> import ffm2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: ./ffm2.so: undefined symbol: _ZN6google4base21CheckOpMessageBuilderD1Ev
>>>

which is a reference to: google::base::CheckOpMessageBuilder::~CheckOpMessageBuilder()

caos21 commented 7 years ago

and it seems to works:

$ pip install . --user
Processing /home/ben/src/fastFM
Requirement already satisfied: numpy in /home/ben/.local/lib/python2.7/site-packages (from fastFM==0.2.9)
Requirement already satisfied: scikit-learn in /home/ben/.local/lib/python2.7/site-packages (from fastFM==0.2.9)
Requirement already satisfied: scipy in /home/ben/.local/lib/python2.7/site-packages (from fastFM==0.2.9)
Installing collected packages: fastFM
  Running setup.py install for fastFM ... done
Successfully installed fastFM-0.2.9

$ ipython
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
Type "copyright", "credits" or "license" for more information.

IPython 2.4.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import ffm

In [2]: import ffm2

In [3]: dir(ffm2)
Out[3]: 
['__builtins__',
 '__doc__',
 '__file__',
 '__name__',
 '__package__',
 '__test__',
 'ffm2_predict',
 'np']

At least I got no errors. It means that is necessary to add a config.cmake in hunter and specify in the CMakeLists.txt:

include("cmake/HunterGate.cmake")
HunterGate(
    URL "https://github.com/ruslo/hunter/archive/v0.17.5.tar.gz"
    SHA1 "033a6ff2cd804144ad55a990bcd631a838ab5d32"
    LOCAL
)

project(fastFM)

hunter_add_package(glog)
hunter_add_package(Eigen)
hunter_add_package(Protobuf)
hunter_add_package(GTest)
hunter_add_package(gflags)
caos21 commented 7 years ago

You will need cmake.config in fastFM2/cmake/Hunter:

#
option(HUNTER_STATUS_DEBUG "Hunter debug info" ON)
hunter_config(glog VERSION 0.3.4-p1)
hunter_config(glog VERSION 0.3.4-p1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(Eigen VERSION 3.3.1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(Protobuf VERSION 3.0.0-p1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(GTest VERSION 1.8.0-hunter-p5 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)
hunter_config(gflags VERSION 2.1.2-p1 CMAKE_ARGS CMAKE_VERBOSE_MAKEFILE=ON CMAKE_POSITION_INDEPENDENT_CODE=ON)

the first 40 lines of CMakeLists.txt as follows:

cmake_minimum_required(VERSION 3.0)

option(FASTFM_GCC_MUSL "Build with GCC musl libc toolchain" OFF)
if(FASTFM_GCC_MUSL)
  set(
      CMAKE_TOOLCHAIN_FILE
      "${CMAKE_CURRENT_LIST_DIR}/cmake/toolchains/gcc-musl.cmake"
      CACHE
      FILEPATH
      ""
      FORCE
  )
endif()

include("cmake/HunterGate.cmake")
HunterGate(
    URL "https://github.com/ruslo/hunter/archive/v0.17.5.tar.gz"
    SHA1 "033a6ff2cd804144ad55a990bcd631a838ab5d32"
    LOCAL
)

project(fastFM)

hunter_add_package(glog)
hunter_add_package(Eigen)
hunter_add_package(Protobuf)
hunter_add_package(GTest)
hunter_add_package(gflags)

option(FASTFM_WITH_GPERFTOOLS "Build with 'gperftools'" OFF)

string(COMPARE EQUAL "${CMAKE_TOOLCHAIN_FILE}" "" no_toolchain)

if(NOT MSVC AND no_toolchain)
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
endif()

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fPIC")

I also moved .hunter directory to another location to enforce the compilation. And you need to change the path of the hunter libs and headers in setup.py.

ibayer commented 7 years ago

I'm not fully able to grasp the implication of your last couple posts but it's looks definitely like we are making good progress. Here are some thought that might help to sort out the build/library path confusion.

ibayer commented 7 years ago

Can you open a PR against fastFM2 with you suggested cmake changes?

ibayer commented 7 years ago

If you want to know the root of Hunter installation directory - you have to build project first (question: https://github.com/ibayer/fastFM/pull/94#issuecomment-299328262). Root is dynamic and depends on a lot of parameters. It can be read from '${CMAKE_BINARY_DIR}/_3rdParty/Hunter/install-root-dir' after build done, i.e. in our case it will be '_builds/_3rdParty/Hunter/install-root-dir'.

caos21 commented 7 years ago

Yes I think we started to see some light :smile:

caos21 commented 7 years ago

at the end setup.py links ffm2.so dynamically against:

$ ldd /home/ben/./.local/lib/python2.7/site-packages/ffm2.so
        linux-vdso.so.1 =>  (0x00007ffcdaeca000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9caf7e0000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9caf5c8000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9caf3a8000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9caefd8000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9caecc8000)
        /lib64/ld-linux-x86-64.so.2 (0x0000559e86c6a000)
ibayer commented 7 years ago

Thanks for the explanations, you should now have access to the server ssh root@46.101.114.187.

I tested if could compile the static libs on the server and then link them locally, but no luck so far :-/ (I copied /home/root/release/to my local machine).

ruslo commented 7 years ago

I think it is doable by creating a config.cmake with the info

You can add it to toolchain.

I also moved .hunter directory to another location to enforce the compilation

If everything is done correctly you don't have to. Local config creates independent Config-ID. If we are talking about modifying toolchain - it will create independent Toolchain-ID. They all will live friendly in one ~/.hunter directory.

Though you have to remove your local build directory: rm -rf _builds.

lets make binary releases for fastFM2 and upload them to github

If you want to speed up build you can use binaries feature:

caos21 commented 7 years ago

Thanks all of you. I will try later. With the config.cmake works, I will look into the toolchain.

And yes no need to move .hunter.

caos21 commented 7 years ago

I am very sorry, please what do you want for me to do in that machine?

very busy, but wanting to follow the issue :man_factory_worker:

ibayer commented 7 years ago

I just wanted to give to access to another machine so that you can test stuff. Please don't feel rushed, I'm just trying to support you with this issue wherever I can.

caos21 commented 7 years ago

Ok thanks. I'm very happy to work in this issue (and in the project), sorry because I will have 3 busy weeks. I will keep in touch, I think we are close to solve this issue and move on.

caos21 commented 7 years ago

I'm back :smiley: catching up on the issue.

ibayer commented 7 years ago

Glad to hear that!

2017-05-19 16:24 GMT+02:00 Benjamin notifications@github.com:

I'm back 😃 catching up on the issue.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ibayer/fastFM/pull/94#issuecomment-302716960, or mute the thread https://github.com/notifications/unsubscribe-auth/ABbYNCSvbeQtf4OlBYqfl4cuz_uj3NyMks5r7aYzgaJpZM4NB31c .

ibayer commented 7 years ago

Hi, can you summarize what challenge you are working on at the moment and what next steps you have planned? I want to catch up with this issue too. :smile:

caos21 commented 7 years ago

Hi, I was checking the procedure and it works. Did you tested it? Also I would like to try with the gcc-muscl build.

caos21 commented 7 years ago

Hi, I tried the following:

  1. Compile with GCC MUSCL. :white_check_mark:
  2. Build and test python package fastFM2. :white_check_mark:
  3. Create a dir /Install with the contents of lib and includefrom .hunter. :white_check_mark:
  4. In the same directory copy the fastFM2 lib and include. :white_check_mark:
  5. In the same vbox but in a different place clone fastFM and build it. Where setup.py pointing to the new dir Install. :white_check_mark:
  6. Test: import the library in ipython :white_check_mark:
  7. In other machine different distro repeat 5 :white_check_mark:
  8. Test fails :beetle: 
In [2]: import ffm2                                                        
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-a8f0c8d8c610> in <module>()                               
----> 1 import ffm2                                                        

ImportError: /home/ben/src/fastFM/ffm2.cpython-35m-x86_64-linux-gnu.so: undefined symbol: 
_ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE

Demangled: 

std::__cxx11::basic_stringbuf<charstd::char_traits<char>std::allocator<char> >

which is in std.

I need this to work before retest in the machine you give me access.

I checked:

$ ldd libfastFM.a 
        not a dynamic executable

I will continue on this.

ibayer commented 7 years ago

Hi, thanks for the update!

Do the tests fail for (4, 5) ? Can you try to compile & run https://gist.github.com/ibayer/7484fa620de10228ab4473453b1fe38f for every setting in which the tests fail?

caos21 commented 7 years ago

They all work, it is only when importing in python that gives the aforementioned error. And I figured out the cause. In my main system I'm using Anaconda, and it uses Anaconda libs, as you can see in the output when building the package:

g++ -pthread -shared -L/home/ben/anaconda3/lib -Wl,-rpath=/home/ben/anaconda3/lib,--no-as-needed build/temp.linux -x86_64-3.5/fastFM/ffm2.o 
-LfastFM2/bmuscl/fastFM -LInstall/lib/fastFM -LInstall/lib -L/home/ben/anaconda3/lib -lfastFM -lpthread -lglog -lpython3.5m 
-o /home/ben/src/fastFM/ffm2.cpython-35m-x86_64-linux-gnu.so

I changed the python path to use python system wide installation and it works!

g++ -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro -Wl,-O1,--sort-common,--as-needed,-z,relro build/temp.linux-x86_64-3.6/fastFM/ffm2.o 
-LfastFM2/bmuscl/fastFM -LInstall/lib/fastFM -LInstall/lib -L/usr/lib -lfastFM -lglog -lpython3.6m -o /home/ben/src/fastFM/ffm2.cpython-36m-x86_64-linux-gnu.so 
-std=c++11

Also your snippet compiles and works in my two systems. Thanks, that suggested that the problem was in the last step of setup.

ibayer commented 7 years ago

Awesome, very glad to hear that it seems to work now. I booted a test server that you can access for further testing ssh root@139.59.147.41.

The final step is now to fix the continuous-integration. We should upload the fastFM2 binaries and modify Travis-ci to include them. All assuming not other issues pop up.

I can have a look at this over the weekend. Please push all your changes. Did you have to modify the GCC MUSCL build steps?

caos21 commented 7 years ago

Thanks, I will repeat the steps in your server.

An intermediate step will be to work with the toolchain as @ruslo suggested. I do not know exactly if it is necessary or desirable or does not matter.

I think I did the MUSCL build as per instructions. But I will need to redo all before pushing changes, I want to be sure.

Sorry, for Travis-ci I do not have idea. Need to study, never used it.

caos21 commented 7 years ago

Hi! Sorry, I have an authentication problem:

$ ssh root@139.59.147.41                                                                                         │
Permission denied (publickey).

Please could you add this key to your server?:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOpccVPq6dKjgoT/EkA6gNdRTZUzWrj5H5bngWZ5bgP83kWjpb9c+I7/FOcC/eG8Grsh5xGjSqzPWbziQqJBzf5EzqgoJ/GBfLZ08ZZ/+TGF+V4u09Uzpk/hMdUNGBUVJhFnUgdbZXg9lGVL3VY11GmmmIii4WV1nYhhcBoClHiFGp2lKGHX1gIFxSKarH42BS6LfaJbeo227jhzl3qGWnSa+pbUFJ10IqPx6RDNSc/o7KBf5KUQ3+QxALUpVffF43DBQrVtrxp2YUhS9ncN/l1f5DUcxJEHwNpNiouuQOfcJEw4DufvO46fNRWUiRHCnn661SP2u9uDz5kDo/hiPp ben@caos21

Thanks

ibayer commented 7 years ago

Hi, I have replaced the key that you gave me in https://github.com/ibayer/fastFM/pull/94#issuecomment-299638572 with your new one.

ibayer commented 7 years ago

I have implemented the fit function for the c++ api. This required some small changes that might effect the cython wrapper. Please let me know if it's okay to merge my changes or if we should wait till this issue is completed. https://github.com/ibayer/fastFM2/pull/18

caos21 commented 7 years ago

Hi, thanks I was able to log in. My fault I think I messed up the keys. I'm catching up.

So the fit function is working? I see you moved some options to the MUSCL toolchain.

What's next? How I could be of any help? I have some hours the day after tomorrow.

ibayer commented 7 years ago

Tomorrow I will decommission the 139.59.147.41 server and spin up a new one. I have cleaned up the musl build and created an ansible playbook (see README) for branch `api_add_fit'. https://github.com/ibayer/fastFM2/tree/api_add_fit

ibayer commented 7 years ago

New test server that you can use (I have added your ssh keys).

root@207.154.217.43

I'm running the ansible playbook just now to setup the musl gcc stuff. Should be ready in about an hour.

ibayer commented 7 years ago

What's next? How I could be of any help? I have some hours the day after tomorrow.

Sorry, I somehow overlook your comment.

The remaining steps are:

I can do the last step. Please let me know if I can help with the others.

ibayer commented 7 years ago

@caos21 Can you give me an estimate if and when you can help me to take over this issue? The results you got a much to good to keep them laying unused.

caos21 commented 7 years ago

@ibayer I'm sorry to keep you waiting, I'm on it today.

caos21 commented 7 years ago

@ibayer

  1. I have to copy your branch api_add_fit into the fastFM directory. For some reason cython doesn't resolve the paths if fastFM2 is outside of fastFM. I think I noted this in the past.

  2. You can check my branch in fastFM dir. It works:

    root@ubuntu-16:~/fastFM (cython-wrapper-#10 *%)# python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 16
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.                                          
>>> import ffm
>>> import ffm2
>>> dir(ffm2)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__test__', 'ffm2_predict', 'np']

  3. The only step I did modify was pip install . --user

Please take a look and check, is it all of this on track?

Note that I didn't build fastFM2 lib, I used your branch api_add_fit.

ibayer commented 7 years ago

Thanks for taking the time. I got 2 errors when running the tests, any hints what I should look for? This might come from the small changes I did to the api.

root@ubuntu-16:~/fastFM (cython-wrapper-#10 *%)# nosetests
..../usr/local/lib/python2.7/dist-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)
........EE.................
======================================================================
ERROR: test_ffm.test_ffm2_predict
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/fastFM/fastFM/tests/test_ffm.py", line 31, in test_ffm2_predict
    y_pred = ffm2.ffm_predict(w0, w, V, X)
AttributeError: 'module' object has no attribute 'ffm_predict'

======================================================================
ERROR: test_ffm.test_ffm2_predict_w0
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/fastFM/fastFM/tests/test_ffm.py", line 38, in test_ffm2_predict_w0
    y_pred = ffm2.ffm_predict(w0, w, V, X)
AttributeError: 'module' object has no attribute 'ffm_predict'

----------------------------------------------------------------------
Ran 31 tests in 0.662s

FAILED (errors=2)
root@ubuntu-16:~/fastFM (cython-wrapper-#10 *%)#
caos21 commented 7 years ago

I think this line

File "/root/fastFM/fastFM/tests/test_ffm.py", line 31, in test_ffm2_predict
    y_pred = ffm2.ffm_predict(w0, w, V, X)

must be

y_pred = ffm2.ffm2_predict(w0, w, V, X)

there is a 2 missing in ffm2_predict

ibayer commented 7 years ago

Your are right, after fixing this I get:

...
7fd9232fb000-7fd9232ff000 r--p 001c0000 fd:01 56060                      /lib/x86_64-linux-gnu/libc-2.23.so
7fd9232ff000-7fd923301000 rw-p 001c4000 fd:01 56060                      /lib/x86_64-linux-gnu/libc-2.23.so
7fd923301000-7fd923305000 rw-p 00000000 00:00 0 
7fd923305000-7fd92331d000 r-xp 00000000 fd:01 29161                      /lib/x86_64-linux-gnu/libpthread-2.23.so
7fd92331d000-7fd92351c000 ---p 00018000 fd:01 29161                      /lib/x86_64-linux-gnu/libpthread-2.23.so
7fd92351c000-7fd92351d000 r--p 00017000 fd:01 29161                      /lib/x86_64-linux-gnu/libpthread-2.23.so
7fd92351d000-7fd92351e000 rw-p 00018000 fd:01 29161                      /lib/x86_64-linux-gnu/libpthread-2.23.so
7fd92351e000-7fd923522000 rw-p 00000000 00:00 0 
7fd923522000-7fd923548000 r-xp 00000000 fd:01 29156                      /lib/x86_64-linux-gnu/ld-2.23.so
7fd923548000-7fd923648000 rw-p 00000000 00:00 0 
7fd923679000-7fd92373e000 rw-p 00000000 00:00 0 
7fd923743000-7fd923744000 rw-p 00000000 00:00 0 
7fd923744000-7fd923745000 rwxp 00000000 00:00 0 
7fd923745000-7fd923747000 rw-p 00000000 00:00 0 
7fd923747000-7fd923748000 r--p 00025000 fd:01 29156                      /lib/x86_64-linux-gnu/ld-2.23.so
7fd923748000-7fd923749000 rw-p 00026000 fd:01 29156                      /lib/x86_64-linux-gnu/ld-2.23.so
7fd923749000-7fd92374a000 rw-p 00000000 00:00 0 
7fff229b4000-7fff229d5000 rw-p 00000000 00:00 0                          [stack]
7fff229ee000-7fff229f0000 r--p 00000000 00:00 0                          [vvar]
7fff229f0000-7fff229f2000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)
root@ubuntu-16:~/fastFM/fastFM/tests (cython-wrapper-#10 *)#
caos21 commented 7 years ago

ok I forgot pthread . One minute

caos21 commented 7 years ago

now I got an Eigen related error.

ibayer commented 7 years ago

now I got an Eigen related error.

running the tests nosetests or building/installing (I get the same error when running the tests)? I don't see why we need Eigen at all in setup.py the lib should run without it as tested with the small cpp test client.

caos21 commented 7 years ago

True. Checking...

caos21 commented 7 years ago

And lglog is needed. All the symbols should be in the ffm2 lib. I don't get it.

ibayer commented 7 years ago

'ffm2.so` is dynamic/shared.

ibayer commented 7 years ago

You mean lglog should be in fastFMd?

caos21 commented 7 years ago

We have,

root@ubuntu-16:~/fastFM/fastFM2/_builds/fastFM (api_add_fit %)# ldd libfastFMd.a
        not a dynamic executable

I thought that we can use it in cython without dependencies, it was the point. But if I don' t link against lglog I have the following error:

ImportError: /root/fastFM/ffm2.so: undefined symbol: _ZN6google4base21CheckOpMessageBuilderC1EPKc

which is 

google::base::CheckOpMessageBuilder::CheckOpMessageBuilder(char const*)

from lglog

ibayer commented 7 years ago

Not clear why we have to do it but it was also the case for the cpp test client.

I thought that we can use it in cython without dependencies, it was the point.

We can easily ship the static lglog with fastFMd, no problem there. This still means we don't have system dependencies which we would have with dynamic linking glog and especially glibc.