Ubuntu: cmake issue - "cannot find -lpthreads"

Coastal0 commented 7 years ago

Hi again,

I've installed a virtualbox with Ubuntu install to try and get pyGimli working, and can't get past the cmake stage of the install instructions.

The 'CMakeError,log" suggests the '-lpthreads' is missing, and I've tried reinstalling and updating all the dependencies I could identify...

Sorry if it's a dumb question again.

florian-wagner commented 7 years ago

Hi @Coastal0,

could you paste the output of the "Dependencies found" section?

Cheers Florian

P.S.: A nice and easy alternative on Linux would be to use the Anaconda Python distribution, then you can just conda install -c gimli pygimli (see http://pygimli.org/installation.html#install-with-conda).

Coastal0 commented 7 years ago

Where would that section be..?

The conda install was my first port of call, but that didn't work either. A problem with python 3.6?

UnsatisfiableError: The following specifications were found to be in conflict:
- pygimli -> boost ==1.60 -> python 2.7* -> openssl 1.0.1*
- python 3.6*
Use "conda info <package>" to see the dependencies for each package

Edit: Here's the CMakeError.log

Determining if the pthread_create exist failed with the following output:
Change Dir: /src/gimli/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_49c1b/fast"
/usr/bin/make -f CMakeFiles/cmTC_49c1b.dir/build.make CMakeFiles/cmTC_49c1b.dir/build
make[1]: Entering directory '/src/gimli/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_49c1b.dir/CheckSymbolExists.c.o
/usr/bin/cc     -o CMakeFiles/cmTC_49c1b.dir/CheckSymbolExists.c.o   -c /src/gimli/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c
Linking C executable cmTC_49c1b
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_49c1b.dir/link.txt --verbose=1
/usr/bin/cc       CMakeFiles/cmTC_49c1b.dir/CheckSymbolExists.c.o  -o cmTC_49c1b -rdynamic 
CMakeFiles/cmTC_49c1b.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x16): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_49c1b.dir/build.make:97: recipe for target 'cmTC_49c1b' failed
make[1]: *** [cmTC_49c1b] Error 1
make[1]: Leaving directory '/src/gimli/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_49c1b/fast' failed
make: *** [cmTC_49c1b/fast] Error 2

File /src/gimli/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef pthread_create
  return ((int*)(&pthread_create))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /src/gimli/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_83d4d/fast"
/usr/bin/make -f CMakeFiles/cmTC_83d4d.dir/build.make CMakeFiles/cmTC_83d4d.dir/build
make[1]: Entering directory '/src/gimli/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_83d4d.dir/CheckFunctionExists.c.o
/usr/bin/cc    -DCHECK_FUNCTION_EXISTS=pthread_create   -o CMakeFiles/cmTC_83d4d.dir/CheckFunctionExists.c.o   -c /usr/share/cmake-3.5/Modules/CheckFunctionExists.c
Linking C executable cmTC_83d4d
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_83d4d.dir/link.txt --verbose=1
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=pthread_create    CMakeFiles/cmTC_83d4d.dir/CheckFunctionExists.c.o  -o cmTC_83d4d -rdynamic -lpthreads 
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_83d4d.dir/build.make:97: recipe for target 'cmTC_83d4d' failed
make[1]: *** [cmTC_83d4d] Error 1
make[1]: Leaving directory '/src/gimli/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_83d4d/fast' failed
make: *** [cmTC_83d4d/fast] Error 2

Coastal0 commented 7 years ago

For reference, I have;

alex@alex-VirtualBox:/home/src/build$ sudo apt-get install subversion git cmake mercurial
Reading package lists... Done
Building dependency tree       
Reading state information... Done
git is already the newest version (1:2.7.4-0ubuntu1).
subversion is already the newest version (1.9.3-2ubuntu1).
mercurial is already the newest version (3.7.3-1ubuntu1).
cmake is already the newest version (3.5.1-1ubuntu3).
0 to upgrade, 0 to newly install, 0 to remove and 5 not to upgrade.

alex@alex-VirtualBox:/home/src/build$ sudo apt-get install libboost-all-dev libblas-dev liblapack-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libblas-dev is already the newest version (3.6.0-2ubuntu2).
liblapack-dev is already the newest version (3.6.0-2ubuntu2).
libboost-all-dev is already the newest version (1.58.0.1ubuntu1).
0 to upgrade, 0 to newly install, 0 to remove and 5 not to upgrade.

alex@alex-VirtualBox:/home/src/build$ sudo apt-get install python-numpy python-matplotlib
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python-numpy is already the newest version (1:1.11.0-1ubuntu1).
python-matplotlib is already the newest version (1.5.1-1ubuntu1).
0 to upgrade, 0 to newly install, 0 to remove and 5 not to upgrade.

alex@alex-VirtualBox:/home/src/build$ sudo apt-get install libedit-dev clang-3.6-dev llvm-3.6-dev python3-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libclang-3.6-dev' for regex 'clang-3.6-dev'
libedit-dev is already the newest version (3.1-20150325-1ubuntu2).
llvm-3.6-dev is already the newest version (1:3.6.2-3ubuntu2).
python3-dev is already the newest version (3.5.1-3).
libclang-3.6-dev is already the newest version (1:3.6.2-3ubuntu2).
0 to upgrade, 0 to newly install, 0 to remove and 5 not to upgrade.

I don't think I've missed any of the things for the tool-chain...

florian-wagner commented 7 years ago

@Coastal0 : Python 3.6 just came out a few weeks ago, and we haven't had the time to build conda packages for pygimli and python 3.6. However, you can use Python 3.5 (or 3.4 and 2.7, but Python 3 is generally recommended) by:

conda install -c gimli python=3.5 pygimli

Coastal0 commented 7 years ago

Thanks florian - it got further, but I think anaconda is having issues linking something scipy...

ERROR conda.core.link:_execute_actions(318): An error occurred while installing package 'defaults::scipy-0.18.1-np111py35_1'.
CondaError: Cannot link a source that does not exist. /home/alex/anaconda3/pkgs/scipy-0.18.1-np111py35_1/lib/python3.5/site-packages/scipy/sparse/base.py

florian-wagner commented 7 years ago

Hi Alex,

hm, this is not gimli related though. Scipy is actually not needed (just for a few functions), so you should be able to work with pygimli already. Are you?

Coastal0 commented 7 years ago

Yeah, I didn't think it would be related, but if you have any suggestions, I'd appreciate them nonetheless!!

I have not been able to get anywhere with it today. Will try again tomorrow...

florian-wagner commented 7 years ago

Okay. You could just try one of the examples on www.pygimli.org, to see if everything works as expected. Even if it works, it would be interesting to know, why your manual compilation failed. May that be due to a missing pthread library (sudo apt-get install libpthread-stubs0-dev)?

Cheers Florian

carsten-forty2 commented 7 years ago

regarding the missing -lpthreads

I don't have access to a ubuntu system to check myself .. can you please try to install sudo apt-get instal libc-dev

I'll try to increase the cmake checks for this case.

Ca

Coastal0 commented 7 years ago

Scipy is actually not needed (just for a few functions), so you should be able to work with pygimli already. Are you?

I'm not sure how I can run it if it didn't compile properly sorry. Just in case, I tried the example python -c 'import pygimli as pg; print(pg._version_)' and got; ModuleNotFoundError: No module named 'pygimli'

re: carsten, apparently it's already installed...

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libc6-dev' instead of 'libc-dev'
libc6-dev is already the newest version (2.23-0ubuntu5).
The following packages were automatically installed and are no longer required:
  linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic linux-image-extra-4.4.0-31-generic
Use 'sudo apt autoremove' to remove them.
0 to upgrade, 0 to newly install, 0 to remove and 29 not to upgrade.

It couldn't find CASTERXML, PYGCCXML, PYPLUSPLUS when I tried to manually reinstall.

-- CASTER_FOUND : Caster: CASTXML_EXECUTABLE-NOTFOUND -- PYGCCXML_FOUND : PYGCCXML: PYGCCXML_PATH-NOTFOUND -- PYPLUSPLUS_FOUND : PYPLUSPLUS_PATH: PYPLUSPLUS_PATH-NOTFOUND

In the "Dependencies" section of your installation guide, these three module are suggested to be installed by the build scripts and so shouldn't installed system wide. I wouldn't know how to install them system-wide anyway (yet).

could you paste the output of the "Dependencies found" section?

I just realised where that was. doh! :(

florian-wagner commented 7 years ago

1. Anaconda installation

@Coastal0 I was under the impression that the conda install -c gimli python=3.5 pygimli worked fine, because the error message you posted was just scipy-related. So you should be able to use pygimli and run examples if you set your PATH accordingly, i.e. export PATH=/home/alex/anaconda3/bin:$PATH (put this to your .bashrc for permanent use).

2. Manual compilation from source

You could try the following:

# Install castxml from binary to avoid any clang/llvm related issues
wget https://midas3.kitware.com/midas/download/item/318227/castxml-linux.tar.gz
tar -xzf castxml-linux.tar.gz
export CASTXML=`pwd`/castxml/bin/castxml
CMAKE_FLAGS="-DCASTER_EXECUTABLE=$CASTXML"

# Install pygimli
mkdir -p gimli
cd gimli
git clone https://github.com/gimli-org/gimli
mkdir -p build
cd build
cmake $CMAKE_FLAGS ../gimli

Important note

Approaches 1 & 2 should not be mixed. So if you use anaconda anyways for your Python stack, than stick with version 1. If you have not used anaconda before and do not want to, than stick with approach 2, but then anaconda (or at least the PATH pointing to it) should be removed. Otherwise system-wide and anaconda libraries may mix, which is very error-prone.

Side note

@gimli-org/gimli-developers I think this is another example, where #52 could be useful. Compiling castxml seems to cause problems on many systems and if the binaries work, why not make it a default rather than an additional cmake flag?

Coastal0 commented 7 years ago

I formatted the disk, started all from scratch and installed the version of mini-conda suggested on the installation page.

It seems to be fine now.

I'm thinking there might have been some conflict with the newer/full version of anaconda...?

Thanks anyway though - I'm sure I'll be in touch again soon... >_>

gimli-org / gimli