search

lab-cosmo / librascal

A scalable and versatile library to generate representations for atomic-scale learning
https://lab-cosmo.github.io/librascal/
GNU Lesser General Public License v2.1
80 stars 20 forks source link

installation questions #348

Closed gabor1 closed 3 years ago

gabor1 commented 3 years ago

hi!

newbie question (first time installation). I'm trying to install librascal into the quip docker. python requirements went in fine, but then pip install . resulted in this. any suggestions? 

docker:librascal$ pip install .
Processing /gabor/git-repos/librascal
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: matplotlib>=2.0.0 in /usr/local/lib/python3.7/site-packages (from rascal==0.0.0) (3.4.1)
Requirement already satisfied: numpy>=1.16.0 in /usr/local/lib/python3.7/site-packages (from rascal==0.0.0) (1.20.2)
Requirement already satisfied: ase>=3.19.0 in /usr/local/lib/python3.7/site-packages (from rascal==0.0.0) (3.22.0b1)
Requirement already satisfied: scipy>=1.2.0 in /usr/local/lib/python3.7/site-packages (from rascal==0.0.0) (1.6.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/site-packages (from matplotlib>=2.0.0->rascal==0.0.0) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/site-packages (from matplotlib>=2.0.0->rascal==0.0.0) (0.10.0)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.7/site-packages (from matplotlib>=2.0.0->rascal==0.0.0) (2.4.7)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.7/site-packages (from matplotlib>=2.0.0->rascal==0.0.0) (8.2.0)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.7/site-packages (from matplotlib>=2.0.0->rascal==0.0.0) (2.8.1)
Requirement already satisfied: six in /usr/local/lib/python3.7/site-packages (from cycler>=0.10->matplotlib>=2.0.0->rascal==0.0.0) (1.15.0)
Building wheels for collected packages: rascal
  Building wheel for rascal (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpmjck4r_v
       cwd: /tmp/pip-req-build-zkmgvttu
  Complete output (94 lines):
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 8.3.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 8.3.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-req-build-zkmgvttu/_cmake_test_compile/build
  -- Build type is: Release
  -- The CXX compiler identification is GNU 8.3.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Downloading wigxjpf
  -- Downloading Eigen3
  -- Eigen3 3.3.4
  -- Found PythonInterp: /usr/local/bin/python (found suitable version "3.7.10", minimum required is "3")
  -- Found PythonLibs: /usr/local/lib/libpython3.7m.so
  -- Downloading pybind11
  -- pybind11 v2.3.0
  -- Installation ROOT: /tmp/pip-req-build-zkmgvttu/_skbuild/linux-x86_64-3.7/cmake-install
  -- The project is built using scikit-build
  -- cpplint parser: /usr/local/bin/cpplint
  -- The optional clang-format auto formatter has not been found. For more information see https://clang.llvm.org/docs/ClangFormat.html
  -- BLACK auto formatter: /usr/local/bin/black
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-req-build-zkmgvttu/_skbuild/linux-x86_64-3.7/cmake-build
  [1/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/units.cc.o
  [2/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/utils.cc.o
  [3/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/gauss_legendre.cc.o
  [4/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/hyp1f1.cc.o
  [5/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/bessel.cc.o
  FAILED: src/CMakeFiles/rascal.dir/rascal/math/bessel.cc.o
  /usr/bin/c++ -Drascal_EXPORTS -I../../../src -isystem external/Eigen3 -isystem external/wigxjpf/inc -isystem external/wigxjpf/cfg -Wall -Wextra -Weffc++ -Wno-non-virtual-dtor -O3 -DNDEBUG -march=native -fPIC -Werror -std=gnu++14 -MD -MT src/CMakeFiles/rascal.dir/rascal/math/bessel.cc.o -MF src/CMakeFiles/rascal.dir/rascal/math/bessel.cc.o.d -o src/CMakeFiles/rascal.dir/rascal/math/bessel.cc.o -c ../../../src/rascal/math/bessel.cc
  c++: fatal error: Killed signal terminated program cc1plus
  compilation terminated.
  [6/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/sparsify_fps.cc.o
  [7/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/interpolator.cc.o
  [8/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/structure_managers/structure_manager_lammps.cc.o
  [9/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/spherical_harmonics.cc.o
  [10/21] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/json_io.cc.o
  ninja: build stopped: subcommand failed.
    File "/tmp/pip-build-env-8wek0fy8/overlay/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 589, in setup
      cmkr.make(make_args, env=env)
    File "/tmp/pip-build-env-8wek0fy8/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 507, in make
      os.path.abspath(CMAKE_BUILD_DIR())))

  --------------------------------------------------------------------------------
  -- Trying "Ninja" generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying "Ninja" generator - success
  --------------------------------------------------------------------------------

  Configuring Project
    Working directory:
      /tmp/pip-req-build-zkmgvttu/_skbuild/linux-x86_64-3.7/cmake-build
    Command:
      cmake /tmp/pip-req-build-zkmgvttu -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-req-build-zkmgvttu/_skbuild/linux-x86_64-3.7/cmake-install -DPYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python -DPYTHON_VERSION_STRING:STRING=3.7.10 -DPYTHON_INCLUDE_DIR:PATH=/usr/local/include/python3.7m -DPYTHON_LIBRARY:FILEPATH=/usr/local/lib/libpython3.7m.so -DSKBUILD:BOOL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-8wek0fy8/overlay/lib/python3.7/site-packages/skbuild/resources/cmake -DINSTALL_PATH:STRING=/tmp/pip-req-build-zkmgvttu/_skbuild/linux-x86_64-3.7/cmake-install -DBUILD_EXAMPLES:BOOL=OFF -DCMAKE_BUILD_TYPE:STRING=Release

  Traceback (most recent call last):

  An error occurred while building with CMake.
    Command:
      cmake --build . --target install --config Release --
    Source directory:
      /tmp/pip-req-build-zkmgvttu
    Working directory:
      /tmp/pip-req-build-zkmgvttu/_skbuild/linux-x86_64-3.7/cmake-build
  Please see CMake's output for more information.
  ----------------------------------------
  ERROR: Failed building wheel for rascal
Failed to build rascal
ERROR: Could not build wheels for rascal which use PEP 517 and cannot be installed directly
docker:librascal$
gabor1 commented 3 years ago

I have also tried python setup install and that seems to fail somewhere else:

docker:librascal$ python setup.py install 

--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
-- Configuring incomplete, errors occurred!
See also "/gabor/git-repos/librascal/_cmake_test_compile/build/CMakeFiles/CMakeOutput.log".
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - failure
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
-- Trying "Unix Makefiles" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is GNU 8.3.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /gabor/git-repos/librascal/_cmake_test_compile/build
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Unix Makefiles" generator - success
--------------------------------------------------------------------------------

Configuring Project
  Working directory:
    /gabor/git-repos/librascal/_skbuild/linux-x86_64-3.7/cmake-build
  Command:
    cmake /gabor/git-repos/librascal -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/gabor/git-repos/librascal/_skbuild/linux-x86_64-3.7/cmake-install -DPYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python -DPYTHON_VERSION_STRING:STRING=3.7.10 -DPYTHON_INCLUDE_DIR:PATH=/usr/local/include/python3.7m -DPYTHON_LIBRARY:FILEPATH=/usr/local/lib/libpython3.7m.so -DSKBUILD:BOOL=TRUE -DCMAKE_MODULE_PATH:PATH=/usr/local/lib/python3.7/site-packages/skbuild/resources/cmake -DINSTALL_PATH:STRING=/gabor/git-repos/librascal/_skbuild/linux-x86_64-3.7/cmake-install -DBUILD_EXAMPLES:BOOL=OFF -DCMAKE_BUILD_TYPE:STRING=Release

-- Build type is: Release
-- The CXX compiler identification is GNU 8.3.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Downloading wigxjpf 
-- Downloading Eigen3 
-- Eigen3 3.3.4
-- Found PythonInterp: /usr/local/bin/python (found suitable version "3.7.10", minimum required is "3") 
-- Found PythonLibs: /usr/local/lib/libpython3.7m.so
-- Downloading pybind11 
-- pybind11 v2.3.0
-- Installation ROOT: /gabor/git-repos/librascal/_skbuild/linux-x86_64-3.7/cmake-install
-- The project is built using scikit-build
-- cpplint parser: /usr/local/bin/cpplint
-- The optional clang-format auto formatter has not been found. For more information see https://clang.llvm.org/docs/ClangFormat.html
-- BLACK auto formatter: /usr/local/bin/black
-- Configuring done
-- Generating done
-- Build files have been written to: /gabor/git-repos/librascal/_skbuild/linux-x86_64-3.7/cmake-build
Scanning dependencies of target rascal
[  5%] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/json_io.cc.o
[ 10%] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/units.cc.o
[ 15%] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/utils.cc.o
[ 20%] Building CXX object src/CMakeFiles/rascal.dir/rascal/utils/sparsify_fps.cc.o
[ 25%] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/bessel.cc.o
[ 30%] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/hyp1f1.cc.o
[ 35%] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/interpolator.cc.o
[ 40%] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/gauss_legendre.cc.o
[ 45%] Building CXX object src/CMakeFiles/rascal.dir/rascal/math/spherical_harmonics.cc.o
[ 50%] Building CXX object src/CMakeFiles/rascal.dir/rascal/structure_managers/structure_manager_lammps.cc.o
[ 55%] Building CXX object src/CMakeFiles/rascal.dir/rascal/structure_managers/structure_manager_centers.cc.o
[ 60%] Building CXX object src/CMakeFiles/rascal.dir/rascal/representations/calculator_base.cc.o
[ 65%] Linking CXX shared library librascal.so
[ 65%] Built target rascal
Scanning dependencies of target _rascal
[ 70%] Building CXX object bindings/CMakeFiles/_rascal.dir/bind_py_module.cc.o
[ 75%] Building CXX object bindings/CMakeFiles/_rascal.dir/bind_py_structure_manager.cc.o
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make[2]: *** [bindings/CMakeFiles/_rascal.dir/build.make:76: bindings/CMakeFiles/_rascal.dir/bind_py_structure_manager.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:918: bindings/CMakeFiles/_rascal.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 589, in setup
    cmkr.make(make_args, env=env)
  File "/usr/local/lib/python3.7/site-packages/skbuild/cmaker.py", line 507, in make
    os.path.abspath(CMAKE_BUILD_DIR())))

An error occurred while building with CMake.
  Command:
    cmake --build . --target install --config Release --
  Source directory:
    /gabor/git-repos/librascal
  Working directory:
    /gabor/git-repos/librascal/_skbuild/linux-x86_64-3.7/cmake-build
Please see CMake's output for more information.
docker:librascal$
gabor1 commented 3 years ago

do you by any chance have a built docker in which I can run librascal? I really just need to be able to run a notebook for the chemistry rev regression manuscript...

Luthaf commented 3 years ago

The error looks like the compiler gets killed by the OS while building librascal.

Can you have a look if the compiler is using more memory than available inside your docker container? I never had a proper look at memory requirements while building librascal, but this could be the culprit. The code builds fine on my machine with 16 GB of RAM, but I'm sure 4-8GB should be more than enough.

do you by any chance have a built docker in which I can run librascal?

I don't think so. As far as I know, most of the core developers are not using docker for day to day operations, instead we usually install librascal in virtual environments.

Luthaf commented 3 years ago

The code builds fine on my machine with 16 GB of RAM, but I'm sure 4-8GB should be more than enough.

As an other data point, the code build fine inside docker on the CI, and have 4 GB of RAM available there. Another possible culprit could be that the container is running out of disk space to store compiled code.

gabor1 commented 3 years ago

thanks. I increased the memory, and python setup.py install now worked. pip install . still didn't work, it gave this error, but maybe it's just conflicting with my attempts at using setup.py

 -- Build files have been written to: /tmp/pip-req-build-k_cc1382/_cmake_test_compile/build
  CMake Error: The current CMakeCache.txt directory /tmp/pip-req-build-k_cc1382/_skbuild/linux-x86_64-3.7/cmake-build/CMakeCache.txt is different than the directory /gabor/git-repos/librascal/_skbuild/linux-x86_64-3.7/cmake-build where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
  CMake Error: The source "/tmp/pip-req-build-k_cc1382/CMakeLists.txt" does not match the source "/gabor/git-repos/librascal/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
    File "/tmp/pip-build-env-jc3by0cz/overlay/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 586, in setup
      languages=cmake_languages
    File "/tmp/pip-build-env-jc3by0cz/overlay/lib/python3.7/site-packages/skbuild/cmaker.py", line 240, in configure
      os.path.abspath(CMAKE_BUILD_DIR())))
Luthaf commented 3 years ago

Yes, the pip install error is indeed related to the other build attempts. You can make it go away by removing /gabor/git-repos/librascal/_skbuild/

max-veit commented 3 years ago

FTR we do have a docker that we use to run the CI, but I don't think it has librascal prebuilt -- although the build should succeed in any case using those images!