search

coin-or / Gravity

Mathematical Modeling for Optimization and Machine Learning
https://www.gravityopt.com
BSD 3-Clause "New" or "Revised" License
148 stars 30 forks source link

installation with Mosek #41

Closed shixuan-zhang closed 5 years ago

shixuan-zhang commented 5 years ago

Hi, Just trying to install Gravity with Mosek, get the following error message: src/CMakeFiles/gravity.dir/build.make:158: recipe for target 'src/CMakeFiles/gravity.dir/MosekProgram.cpp.o' failed make[2]: *** [src/CMakeFiles/gravity.dir/MosekProgram.cpp.o] Error 1 CMakeFiles/Makefile2:179: recipe for target 'src/CMakeFiles/gravity.dir/all' failed make[1]: *** [src/CMakeFiles/gravity.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: *** [all] Error 2 Some errors reported by the compiler are of the following form: /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp: In member function ‘void MosekProgram::set_mosek_objective()’: /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp:828:81: error: ‘poly_eval’ was not declared in this scope monty::rc_ptr< ::mosek::fusion::Expression > expr= fusion::Expr::constTerm(poly_eval(_model->_obj.get_cst())); // expr is a pointer to the Expression. ^~~~~~~~~ /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp:828:81: note: suggested alternative: ‘poly_h’ monty::rc_ptr< ::mosek::fusion::Expression > expr= fusion::Expr::constTerm(poly_eval(_model->_obj.get_cst())); // expr is a pointer to the Expression. ^~~~~~~~~ poly_h /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp:857:70: error: ‘class gravity::param_’ has no member named ‘get_nb_instances’; did you mean ‘get_id_inst’? auto coefs = new_array_ptr<double,1>((it1.second._p->get_nb_instances())); ^~~~~~~~~~~~~~~~ get_id_inst /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp:858:50: error: ‘class gravity::param_’ has no member named ‘get_nb_instances’; did you mean ‘get_id_inst’? for (int j = 0; j<it1.second._p->get_nb_instances(); j++) { ^~~~~~~~~~~~~~~~ get_id_inst /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp:878:70: error: ‘class gravity::param_’ has no member named ‘get_nb_instances’; did you mean ‘get_id_inst’? auto coefs = new_array_ptr<double,1>((it1.second._p->get_nb_instances())); ^~~~~~~~~~~~~~~~ get_id_inst /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp:879:50: error: ‘class gravity::param_’ has no member named ‘get_nb_instances’; did you mean ‘get_id_inst’? for (int j = 0; j<it1.second._p->get_nb_instances(); j++) { ^~~~~~~~~~~~~~~~ get_id_inst /home/zhangshixuanus/Gravity/Gravity/src/MosekProgram.cpp:887:61: error: ‘class gravity::param_’ has no member named ‘get_sdpid’; did you mean ‘get_dim’? pair<size_t, size_t> pair = it1.second._p->get_sdpid(); ^~~~~~~~~ get_dim Could you please look into this error? Thank you very much.

hhijazi commented 5 years ago

Mosek is not yet supported in the Master branch, I recommend using the latest stable Release: https://github.com/coin-or/Gravity/releases if you intend to call Mosek.

shixuan-zhang commented 5 years ago

Thank you for the information.

shixuan-zhang commented 5 years ago

I downloaded the stable Release. During the make process, the following error occurs: [ 36%] Linking CXX executable ../../bin/socopf ../../lib/libgravity.so: undefined reference to 'pthread_create' collect2: error: ld returned 1 exit status examples/CMakeFiles/socopf.dir/build.make:255: recipe for target '../bin/socopf' failed make[2]: *** [../bin/socopf] Error 1 CMakeFiles/Makefile2:312: recipe for target 'examples/CMakeFiles/socopf.dir/all' failed make[1]: *** [examples/CMakeFiles/socopf.dir/all] Error 2 Makefile:129: recipe for target 'all' failed make: *** [all] Error 2 Do you have any idea how should I deal with this error? Thank you!

hhijazi commented 5 years ago

this seems to be a missing -pthread flag, try updating the following line in the CMakeLists.txt file at the root: set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11") to set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -pthread")

hhijazi commented 5 years ago

make sure to do rm -fr * in the build directory before rerunning cmake ..

shixuan-zhang commented 5 years ago

Thanks. I have tried add -pthread in the CMakeLists.txt file. I do not see the previous error any more. However, some other error appears: [ 81%] Linking CXX executable ../../bin/sdpopf CMakeFiles/sdpopf.dir/MINLP/Power/SDPOPF/Bag.cpp.o: In function 'bool arma::auxlib::eig_sym_dc<double, arma::Mat<std::complex<double> > >(arma::Col<double>&, arma::Mat<std::complex<double> >&, arma::Base<std::complex<double>, arma::Mat<std::complex<double> > > const&)': Bag.cpp:(.text._ZN4arma6auxlib10eig_sym_dcIdNS_3MatISt7complexIdEEEEEbRNS_3ColIT_EERNS2_IS3_IS7_EEERKNS_4BaseISA_T0_EE[_ZN4arma6auxlib10eig_sym_dcIdNS_3MatISt7complexIdEEEEEbRNS_3ColIT_EERNS2_IS3_IS7_EEERKNS_4BaseISA_T0_EE]+0x1a4): undefined reference to 'zheevd_' CMakeFiles/sdpopf.dir/MINLP/Power/SDPOPF/Bag.cpp.o: In function 'bool arma::eig_sym<arma::Mat<std::complex<double> > >(arma::Col<arma::Mat<std::complex<double> >::pod_type>&, arma::Mat<arma::Mat<std::complex<double> >::elem_type>&, arma::Base<arma::Mat<std::complex<double> >::elem_type, arma::Mat<std::complex<double> > > const&, char const*, arma::arma_blas_type_only<arma::Mat<std::complex<double> >::elem_type>::result const*)': Bag.cpp:(.text._ZN4arma7eig_symINS_3MatISt7complexIdEEEEEbRNS_3ColINT_8pod_typeEEERNS1_INS6_9elem_typeEEERKNS_4BaseISA_S6_EEPKcPKNS_19arma_blas_type_onlyISA_E6resultE[_ZN4arma7eig_symINS_3MatISt7complexIdEEEEEbRNS_3ColINT_8pod_typeEEERNS1_INS6_9elem_typeEEERKNS_4BaseISA_S6_EEPKcPKNS_19arma_blas_type_onlyISA_E6resultE]+0x1ea): undefined reference to 'zheev_' CMakeFiles/sdpopf.dir/MINLP/Power/SDPOPF/Bag.cpp.o: In function 'void arma::herk_emul<false, false, false>::apply<double, arma::Mat<std::complex<double> > >(arma::Mat<std::complex<double> >&, arma::Mat<std::complex<double> > const&, double, double)': Bag.cpp:(.text.hot._ZN4arma9herk_emulILb0ELb0ELb0EE5applyIdNS_3MatISt7complexIdEEEEEvRNS3_IS4_IT_EEERKT0_S7_S7_[_ZN4arma9herk_emulILb0ELb0ELb0EE5applyIdNS_3MatISt7complexIdEEEEEvRNS3_IS4_IT_EEERKT0_S7_S7_]+0x235): undefined reference to 'zgemv_' CMakeFiles/sdpopf.dir/MINLP/Power/SDPOPF/Bag.cpp.o: In function 'void arma::glue_times::apply<std::complex<double>, false, true, false, arma::Mat<std::complex<double> >, arma::Mat<std::complex<double> > >(arma::Mat<std::complex<double> >&, arma::Mat<std::complex<double> > const&, arma::Mat<std::complex<double> > const&, std::complex<double>)': Bag.cpp:(.text.hot._ZN4arma10glue_times5applyISt7complexIdELb0ELb1ELb0ENS_3MatIS3_EES5_EEvRNS4_IT_EERKT3_RKT4_S6_[_ZN4arma10glue_times5applyISt7complexIdELb0ELb1ELb0ENS_3MatIS3_EES5_EEvRNS4_IT_EERKT3_RKT4_S6_]+0x150): undefined reference to 'zgemm_' Bag.cpp:(.text.hot._ZN4arma10glue_times5applyISt7complexIdELb0ELb1ELb0ENS_3MatIS3_EES5_EEvRNS4_IT_EERKT3_RKT4_S6_[_ZN4arma10glue_times5applyISt7complexIdELb0ELb1ELb0ENS_3MatIS3_EES5_EEvRNS4_IT_EERKT3_RKT4_S6_]+0x1e6): undefined reference to 'zherk_' Bag.cpp:(.text.hot._ZN4arma10glue_times5applyISt7complexIdELb0ELb1ELb0ENS_3MatIS3_EES5_EEvRNS4_IT_EERKT3_RKT4_S6_[_ZN4arma10glue_times5applyISt7complexIdELb0ELb1ELb0ENS_3MatIS3_EES5_EEvRNS4_IT_EERKT3_RKT4_S6_]+0x40e): undefined reference to 'zgemv_' collect2: error: ld returned 1 exit status make[2]: *** [examples/CMakeFiles/sdpopf.dir/build.make:192: ../bin/sdpopf] Error 1 make[1]: *** [CMakeFiles/Makefile2:591: examples/CMakeFiles/sdpopf.dir/all] Error 2 make: *** [Makefile:130: all] Error 2 Thanks for the help.

hhijazi commented 5 years ago

Is the cmake build finding and building armadillo? You might try adding those flags: "-larmadillo -llapack -lblas" where you added -pthread.

shixuan-zhang commented 5 years ago

I see. Now it says the it cannot find -larmadillo, -llapack, and -lblas. I will check with the system admin the directories of the libraries, lapack and blas. But is armadillo included in the stable Release or do I need to install it separately and set the environment variable of the path?

hhijazi commented 5 years ago

Can you copy paste the log you get when running "cmake .."

shixuan-zhang commented 5 years ago

Sure, it shows the followings: cmake ..
Armadillo root dir /home/zhangshixuanus/Gravity/Gravity/ext_lib/armadillo-8.100.0 Armadillo version 8.100.0 -- Configuring Armadillo 8.100.0 -- CMAKE_SYSTEM_NAME = Linux -- CMAKE_CXX_COMPILER_ID = GNU -- CMAKE_CXX_COMPILER_VERSION = 7.3.1 -- CMAKE_COMPILER_IS_GNUCXX = 1 -- BUILD_SHARED_LIBS = ON -- DETECT_HDF5 = ON -- MKL_FOUND = NO -- ACMLMP_FOUND = NO -- ACML_FOUND = NO -- OpenBLAS_FOUND = NO -- ATLAS_FOUND = NO -- BLAS_FOUND = NO -- LAPACK_FOUND = NO -- Checking for module 'hdf5' -- No package 'hdf5' found -- HDF5_FOUND = -- ARPACK_FOUND = NO -- Looking for SuperLU version 5 -- Could not find SuperLU -- SuperLU_FOUND = NO -- *** Armadillo wrapper library will use the following libraries: -- *** ARMA_LIBS = -- Detected gcc 4.8.3 or later. Added '-std=c++11' to compiler flags -- Copying /home/zhangshixuanus/Gravity/Gravity/ext_lib/armadillo-8.100.0/include/ to /home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/tmp/include/ -- Generating /home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/tmp/include/config.hpp -- Generating /home/zhangshixuanus/Gravity/Gravity/ext_lib/armadillo-8.100.0/examples/Makefile -- CMAKE_CXX_FLAGS = -std=c++11 -O2 -- CMAKE_SHARED_LINKER_FLAGS = -Wl,--no-as-needed -- CMAKE_REQUIRED_INCLUDES = -- CMAKE_INSTALL_PREFIX = /usr -- CMAKE_INSTALL_LIBDIR = lib64 -- CMAKE_INSTALL_INCLUDEDIR = include -- CMAKE_INSTALL_DATADIR = share -- CMAKE_INSTALL_BINDIR = bin -- Generating '/home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/ArmadilloConfig.cmake' -- Generating '/home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/ArmadilloConfigVersion.cmake' -- Generating '/home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/InstallFiles/ArmadilloConfig.cmake' -- Generating '/home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/InstallFiles/ArmadilloConfigVersion.cmake' -- Copying /home/zhangshixuanus/Gravity/Gravity/ext_lib/armadillo-8.100.0/misc/ to /home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/tmp/misc/ -- Generating '/home/zhangshixuanus/Gravity/Gravity/build/ext_lib/armadillo-8.100.0/tmp/misc/armadillo.pc' -- Configuring done -- Generating done

hhijazi commented 5 years ago

I meant the Gravity cmake log

shixuan-zhang commented 5 years ago

Do you mean this log file? CMakeOutput.log

hhijazi commented 5 years ago

Can you go do the following and send me the log: cd Gravity-1.0.8 2 mkdir new_build cd new_build cmake .. -DENABLE_MOSEK=ON make also remove -lblas and -llapack keep -larmadillo

shixuan-zhang commented 5 years ago

Sure, it seems to have some different error. new_cmake.log

hhijazi commented 5 years ago

Actually the error seems to be in Mosek, did you follow the instructions to build mosek fusion c++ library? If not take a look at Gravity’s Install.md file and follow the links there

hhijazi commented 5 years ago

The compiler was only showing warnings

shixuan-zhang commented 5 years ago

Thanks for the reply. I did follow the Install.md and ran 'make install' to install the fusion library for Mosek. I also thought this might be due to the version difference between Mosek 8 and 9. However, after I switched back to Mosek 8, I still get the same error log. new_cmake.log Do you have any idea what might be the cause? Thank you!

hhijazi commented 5 years ago

Ok, you just need to add: using namespace gravity; at the top of file include/gravity/MosekProgram.h, just before class MosekPrgoram:.

hhijazi commented 5 years ago

Sorry it took so long, the Mosek interface was coded by other contributors.

shixuan-zhang commented 5 years ago

Thanks for the reply. The error does not show up again. Yet there is still some linking error. Could you please look at it? Thank you very much. new_cmake.log

hhijazi commented 5 years ago

In the file cmake/FindMOSEK.cmake on line 31 remove -Wl

shixuan-zhang commented 5 years ago

Thanks for the comment. Now it goes back the the first error I have seen. That is the linking error saying "cannot find -larmadillo" cmake.log

hhijazi commented 5 years ago

try removing -larmadillo

shixuan-zhang commented 5 years ago

It now shows a different linking error. cmake.log

hhijazi commented 5 years ago

Ok, I think the only way to fix this is to update the Mosek interface to be compatible with the current version of Gravity. I will try to do the update today.

shixuan-zhang commented 5 years ago

Thank you very much.

hhijazi commented 5 years ago

Can you try pulling the Complex branch of Gravity and giving it a try?

shixuan-zhang commented 5 years ago

Thank you so much for the work. I pulled the Complex branch and ran the cmake configuration with Mosek option turned on. The compiling process does not have those warnings any more. Yet there seems to be an error while compiling the target obbt: cmake.log Is there anything I didn't configure correctly?

hhijazi commented 5 years ago

No, you just need to pull again, I fixed this just after sending the message.

hhijazi commented 5 years ago

Everything should compile, but I cannot vouch for the Mosek interface being bug-free since I was not involved in its implementation, but since this is an open-source project, contributions are very welcome, if you will be using it extensively, I recommend you try to make sue it works and submit a pull request so I can then integrate your work to the master branch.

hhijazi commented 5 years ago

@zhangshixuanus let me know if I can close this issue.

shixuan-zhang commented 5 years ago

Sorry, but actually there was still some error in compiling. cmake.log This seems to be an example, right? So I am trying to see if I could use Gravity without this example.

hhijazi commented 5 years ago

This is just a missing -pthread flag for linux, I added it now, try to pull and run again. If you want to turn off some of the examples you can edit Gravity/examples/CmakeLists.txt and comment both the add_executable() and target_link_libraries() lines corresponding to that example.

shixuan-zhang commented 5 years ago

There are still some errors with the pthread_create function linking even after adding the -pthread flag for linux. However, after I removed the examples obbt and gravity_test, all the other parts were compiled successfully. Thank you very much for all the help.

hhijazi commented 5 years ago

Closing this issue, pthread-create errors are not showing up on the Travis linux build.