search

cselab / smarties

Lightweight and scalable framework for Reinforcement Learning
MIT License
117 stars 50 forks source link

compiling error on Mac OS #3

Closed waynezw0618 closed 4 years ago

waynezw0618 commented 4 years ago

Hi I followed the instruction of the installation by install pyblinder11 first. but during compiling, still get error message like : -fPIC -c ../source/smarties_pybind11.cpp -o smarties_pybind11.o ../source/smarties_pybind11.cpp:10:10: fatal error: 'pybind11/pybind11.h' file not found

include <pybind11/pybind11.h>

then I use python -m pip install pybind11 to install pybind11 as suggested here . I got the following error message during link mkdir -p ../lib mkdir -p ../lib mpic++ -shared -fPIC -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -o ../lib/libsmarties.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o -L/usr/local/opt/openblas/lib/ -lopenblas mpic++ -shared -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -Wl,-undefined,dynamic_lookup -o ../lib/smarties.cpython-37m-darwin.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o smarties_pybind11.o -L/usr/local/opt/openblas/lib/ -lopenblas ld: unknown option: -platform_version ld: unknown option: -platform_version clang-10clang-10: : error: error: linker command failed with exit code 1 (use -v to see invocation) linker command failed with exit code 1 (use -v to see invocation) make: [py11] Error 1 make: Waiting for unfinished jobs.... make: *** [lib] Error 1

Thanks !

novatig commented 4 years ago

Hi,

What happens if you type "python3 -m pybind11 --includes" ?

waynezw0618 commented 4 years ago

Hi novatig: Thanks for replying.

(python3) my-mbp:makefiles my$ python3 -m pybind11 --include -I/anaconda2/envs/python3/include/python3.7m -I/Users/my/.local/lib/python3.7/site-packages/pybind11/include

and it is now can find the pybind11.h as I use "python -m pip install pybind11" but still some errors for linking.

2 warnings generated. mkdir -p ../lib mkdir -p ../lib mpic++ -shared -fPIC -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -o ../lib/libsmarties.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o -L/usr/local/opt/openblas/lib/ -lopenblas mpic++ -shared -L/usr/local/Cellar/libomp/10.0.0/lib -fopenmp=libomp -Wl,-undefined,dynamic_lookup -o ../lib/smarties.cpython-37m-darwin.so PPO.o MixedPG.o NAF.o DPG.o DQN.o ACER.o RACER.o CMALearner.o Learner_pytorch.o Learner_approximator.o AlgoFactory.o Episode.o Learner.o Sampling.o Collector.o Math_tests.o MemoryBuffer.o DataCoordinator.o MemoryProcessing.o Builder.o Network.o Optimizer.o CMA_Optimizer.o StatsTracker.o Communicator.o Approximator.o Launcher.o Settings.o Profiler.o Master.o Worker.o StateAction.o Engine.o Warnings.o smarties_extern.o smarties_pybind11.o -L/usr/local/opt/openblas/lib/ -lopenblas ld: unknown option: -platform_version ld: unknown option: -platform_version clang-10: error: linker command failed with exit code 1 (use -v to see invocation) clang-10: error: linker command failed with exit code 1 (use -v to see invocation) make: [py11] Error 1 make: Waiting for unfinished jobs.... make: *** [lib] Error 1

novatig commented 4 years ago

I see. Try to compile with cmake (same commands as for linux). Do you get the same error?

waynezw0618 commented 4 years ago

Hi, Novatig thanks very much for replying I follow the that part and use cmake, still does not work me . here is there error message:

(python3) weis-mbp:build my$ /Applications/CMake.app/Contents/bin/cmake ../ -- The C compiler identification is AppleClang 10.0.1.10010046 -- The CXX compiler identification is AppleClang 10.0.1.10010046 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found MPI_C: /usr/local/Cellar/open-mpi/4.0.3/lib/libmpi.dylib (found version "3.1") -- Found MPI_CXX: /usr/local/Cellar/open-mpi/4.0.3/lib/libmpi.dylib (found version "3.1") -- Found MPI: TRUE (found version "3.1")
-- Found OpenMP_C: -Xclang -fopenmp (found version "3.1") -- Found OpenMPCXX: -Xclang -fopenmp (found version "3.1") -- Found OpenMP: TRUE (found version "3.1")
-- Looking for sgemm -- Looking for sgemm_ - not found -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBCPTHREAD - Success -- Found Threads: TRUE
-- Looking for dgemm -- Looking for dgemm_ - found -- Found BLAS: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Accelerate.framework
CMake Error at CMakeLists.txt:138 (add_subdirectory): The source directory

/Users/my/software/ML/smarties/source/extern/pybind11

does not contain a CMakeLists.txt file.

novatig commented 4 years ago

It should: did you git clone --recursive https://github.com/cselab/smarties.git? Or just git clone?

waynezw0618 commented 4 years ago

I used latter. you are right, I can now compile with former. but still error messages from compiling the application (python3) weis-mbp:cart_pole_cpp weizhang$ make mpic++ -std=c++14 -Wall -O2 -I/Users/my/software/ML/smarties//include -Wextra -Wfloat-equal -Wundef -Wcast-align -Wno-unused-parameter -Wwrite-strings -Wlogical-op -Wmissing-declarations -Wredundant-decls -Wshadow -Woverloaded-virtual -Wuninitialized -Wno-div-by-zero -c cart-pole.cpp -o cart-pole.o warning: unknown warning option '-Wlogical-op'; did you mean '-Wlong-long'? [-Wunknown-warning-option] In file included from cart-pole.cpp:9: In file included from /Users/my/software/ML/smarties//include/smarties.h:12: In file included from /Users/my/software/ML/smarties//include/../source/Communicator.h:12: In file included from /Users/my/software/ML/smarties//include/../source/Core/Environment.h:12: /Users/my/software/ML/smarties//include/../source/Core/Agent.h:228:14: warning: cast from 'char ' to 'smarties::episodeStatus ' increases required alignment from 1 to 4 [-Wcast-align] return (episodeStatus ) buffer; ^~~~~~~~ /Users/my/software/ML/smarties//include/../source/Core/Agent.h:233:14: warning: cast from 'char ' to 'smarties::learnerStatus ' increases required alignment from 1 to 4 [-Wcast-align] return (learnerStatus ) buffer; ^~~~~~~~ In file included from cart-pole.cpp:9: In file included from /Users/my/software/ML/smarties//include/smarties.h:12: In file included from /Users/my/software/ML/smarties//include/../source/Communicator.h:13: In file included from /Users/my/software/ML/smarties//include/../source/Utils/MPIUtilities.h:12: /usr/local/Cellar/open-mpi/4.0.3/include/mpi.h:316:16: warning: '__STDC_VERSION__' is not defined, evaluates to 0 [-Wundef]

elif (__STDC_VERSION__ >= 201112L)

           ^

4 warnings generated.

mpic++ -o cart-pole cart-pole.o -L/Users/my/software/ML/smarties//lib -lsmarties ld: unknown option: -platform_version clang-10: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [cart-pole] Error 1

novatig commented 4 years ago

Are you on OS X Mojave (https://github.com/Homebrew/homebrew-core/issues/52461)? Anyway, it seems your cmake works fine. I changed from make to cmake, on the cart_pole_cpp app only (for now). Pull again and see if it works.

waynezw0618 commented 4 years ago

@novtig perfect！it works now. can you please update other apps? I am interested in boatNav or glider

thanks

novatig commented 4 years ago

Hi, I'm very busy at the time and I won't stop being busy till the fall (I'm defending my PhD thesis in August). If you urgently want to try those apps, have a read at the CMakeLists.txt and setup.sh files in apps/cart_pole_cpp/ They are really self-explanatory

waynezw0618 commented 4 years ago

Sorry for troubling. I am trying to shift to ubuntu. still error arises like :

-- Found MPI_C: /usr/lib/x86_64-linux-gnu/libmpich.so
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/libmpichcxx.so;/usr/lib/x86_64-linux-gnu/libmpich.so
-- Try OpenMP C flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAG_DETECTED - Success -- Try OpenMP CXX flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAGDETECTED - Success -- Found OpenMP: -fopenmp
-- Looking for sgemm -- Looking for sgemm_ - found -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
-- A library with BLAS API found. -- Found PythonInterp: /usr/local/bin/python3.7 (found version "3.7.3") -- Found PythonLibs: /usr/local/lib/libpython3.7m.so -- Performing Test HAS_CPP14_FLAG -- Performing Test HAS_CPP14_FLAG - Success -- pybind11 v2.4.dev4 -- Configuring incomplete, errors occurred! See also "/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeOutput.log". See also "/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeError.log". here is information from CMAKEERROR Determining if the pthread_create exist failed with the following output: Change Dir: /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_0fad3/fast" /usr/bin/make -f CMakeFiles/cmTC_0fad3.dir/build.make CMakeFiles/cmTC_0fad3.dir/build make[1]: Entering directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o /usr/bin/cc -o CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o -c /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c Linking C executable cmTC_0fad3 /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_0fad3.dir/link.txt --verbose=1 /usr/bin/cc CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o -o cmTC_0fad3 -rdynamic CMakeFiles/cmTC_0fad3.dir/CheckSymbolExists.c.o: In function main': CheckSymbolExists.c:(.text+0x1b): undefined reference topthread_create' collect2: error: ld returned 1 exit status CMakeFiles/cmTC_0fad3.dir/build.make:97: recipe for target 'cmTC_0fad3' failed make[1]: [cmTC_0fad3] Error 1 make[1]: Leaving directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp' Makefile:126: recipe for target 'cmTC_0fad3/fast' failed make: [cmTC_0fad3/fast] Error 2

File /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c: / /

include

int main(int argc, char** argv) { (void)argv;

ifndef pthread_create

return ((int*)(&pthread_create))[argc];

else

(void)argc; return 0;

endif

}

Determining if the function pthread_create exists in the pthreads failed with the following output: Change Dir: /Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_8cb33/fast" /usr/bin/make -f CMakeFiles/cmTC_8cb33.dir/build.make CMakeFiles/cmTC_8cb33.dir/build make[1]: Entering directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_8cb33.dir/CheckFunctionExists.c.o /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_8cb33.dir/CheckFunctionExists.c.o -c /usr/share/cmake-3.7/Modules/CheckFunctionExists.c Linking C executable cmTC_8cb33 /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_8cb33.dir/link.txt --verbose=1 /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create CMakeFiles/cmTC_8cb33.dir/CheckFunctionExists.c.o -o cmTC_8cb33 -rdynamic -lpthreads /usr/bin/ld: cannot find -lpthreads collect2: error: ld returned 1 exit status CMakeFiles/cmTC_8cb33.dir/build.make:97: recipe for target 'cmTC_8cb33' failed make[1]: [cmTC_8cb33] Error 1 make[1]: Leaving directory '/Users/weizhang/software/ubuntu/smarties/build/CMakeFiles/CMakeTmp' Makefile:126: recipe for target 'cmTC_8cb33/fast' failed make: [cmTC_8cb33/fast] Error 2

would you please help when you have time

novatig commented 4 years ago

This is not an issue with smarties. As always, just look at the first error and ask your best friend: https://www.google.com/search?q=CMAKEERROR+Determining+if+the+pthread_create+exist+failed+with+the+following+output%3A

waynezw0618 commented 4 years ago

Hi Novatig: I understand that you are very busy at the moment. I am currently using your code. I am playing glider and boatNav. I am wondering, how to setup parameters in the json file, for instance, nnLayerSizes, epsAnneal and so on. thanks for taking your time to answer the question. Yours Sincerely

Wei

novatig commented 4 years ago

Hi, sorry for the wait. Thank you for your message, it reminded me one way the README was (is?) lacking. Try reading https://github.com/cselab/smarties/blob/master/README.rst#learner-settings-json-files and let me know any doubt you still have.

However, please open a new ticket and close this one if your doubts are unrelated to compilation.

waynezw0618 commented 4 years ago

@novatig thanks! I have opened a new one (https://github.com/cselab/smarties/issues/4)