search

LuxCoreRender / LinuxCompile

LuxCoreRender Linux Compilation Environment
10 stars 5 forks source link

How to install required dependencies? #9

Closed Theverat closed 4 years ago

Theverat commented 4 years ago

I am currently trying to set up the static compilation environment on Ubuntu 18.04. The readme in this repository is lacking instructions on how to install the required dependencies. Especially Python is a mystery to me: Lux needs another Python version than what is pre-installed by the OS. But I have no idea how to install it correctly.

I tried to compile Python 3.7.0 from source and install it, but apparently it was not done right, because I'm getting errors about missing Python when running the Lux build script. I would like not to go into detail about what I did and what kind of errors I now get - instead, it would be great if someone who has already done this successfully could describe how to download and install the required Python version correctly, with all needed libraries and headers and whatnot in the right places. I will then get my build environment going and if I succeed, I'll fill in this missing information in the readme and wiki.

So far every time I have set up a new build environment for Lux, it was always a long grind of failure after failure, sometimes I gave up for months, sometimes it worked and I didn't know why exactly. This is my first attempt since my laptop broke down, and I'd like to change this in the future.

Dade916 commented 4 years ago

Ubuntu 18.04 has 2 python versions installed: v2.x and v3.x. v3.x is fine both for Lux and Blender. What error do you get when you run the compilation script ?

Theverat commented 4 years ago

It has version 3.6 installed by default: https://launchpad.net/ubuntu/bionic/+package/python3 AFAIK I need the same version as the Blender I'm building for, so 3.7.

I would like to improve the readme/wiki so the steps work even if OS package versions don't match the required versions (which is almost always the case). For example, the default cmake of Ubuntu is also not recent enough to compile LuxCore, so you have to install the latest version manually.

That's why I think general advice would be best, not solving my particular case once (and having errors pop up again when trying the same on a later Ubuntu in a few years).

But if you insist:

./build-64-sse2 LuxCore 1.1
 *** BUILDING FOR X86-64 (SSE2) ***
 * LuxCore static enabled
 * OpenCL disabled
 * Using GCC compiler
 * Checking distfiles
boost_1_67_0.tar.gz
bzip2-1.0.6.tar.gz
c-blosc-1.17.1-bin.tgz
embree-3.2.4.x86_64.linux.tar.gz
freetype-2.4.11.tar.bz2
ilmbase-2.2.0.tar.gz
jpegsrc.v6b.tar.gz
libpng-1.5.26.tar.xz
oidn-1.0.0.x86_64.linux.tar.gz
oiio-Release-1.8.11.tar.gz
openexr-2.2.0.tar.gz
qt-5.12.2.tar.gz
tbb2018_20180312oss_lin.tgz
tiff-4.0.3.tar.gz
xz-5.0.5.tar.xz
zlib-1.2.8.tar.gz
 * Done checking distfiles
 * Checking target tree
 * Target tree exists
 * Checking libstdc++
 * Done checking libstdc++
 * Checking python
 * Done checking python
 * Checking LuxMark scenes
 * Done LuxMark scenes
 * Prechecks completed

 * Using Python library path: /home/simon/Projects/LinuxCompile/target-64-sse2/lib/libpython3.7.a
 * Using Python include path: /usr/include/python3.6m

 * Cleaning and unpacking boost
 * Compiling boost
 * Patching Boost
'/home/simon/Projects/LinuxCompile/distfiles/boost-1.67-numpy.cpp' -> 'libs/python/src/numpy/numpy.cpp'
 * Compiling boost for python-3.7
Building Boost.Build engine with toolset gcc... tools/build/src/engine/bin.linuxx86_64/b2
Detecting Python version... 3.7
Unicode/ICU support for Boost.Regex?... disabled.
Generating Boost.Build configuration in project-config.jam...

Bootstrapping is done. To build, run:

    ./b2

To adjust configuration, edit 'project-config.jam'.
Further information:

   - Command line help:
     ./b2 --help

   - Getting started guide: 
     http://www.boost.org/more/getting_started/unix-variants.html

   - Boost.Build documentation:
     http://www.boost.org/build/doc/html/index.html

/tmp/luxbuild/boost_1_67_0/libs/predef/check/../tools/check/predef.jam:46: Unescaped special character in argument $(language)::$(expression)
Performing configuration checks

    - default address-model    : 64-bit
    - default architecture     : x86
    - symlinks supported       : yes
    - zlib                     : yes
    - bzip2                    : yes
    - lzma                     : no
    - iconv (libc)             : yes
    - icu                      : no
    - icu (lib64)              : no
    - has_icu builds           : no
    - lockfree boost::atomic_flag : yes

Component configuration:

    - atomic                   : not building
    - chrono                   : building
    - container                : not building
    - context                  : not building
    - contract                 : not building
    - coroutine                : not building
    - date_time                : building
    - exception                : not building
    - fiber                    : not building
    - filesystem               : building
    - graph                    : not building
    - graph_parallel           : not building
    - iostreams                : building
    - locale                   : building
    - log                      : not building
    - math                     : not building
    - mpi                      : not building
    - program_options          : building
    - python                   : building
    - random                   : not building
    - regex                    : building
    - serialization            : building
    - signals                  : not building
    - stacktrace               : not building
    - system                   : building
    - test                     : not building
    - thread                   : building
    - timer                    : not building
    - type_erasure             : not building
    - wave                     : not building

...patience...
...patience...
...found 32967 targets...
...updating 13616 targets...
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/chrono/stopwatches.hpp
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/chrono/stopwatches/suspendable_stopwatch.hpp
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/chrono/stopwatches/strict_stopwatch.hpp
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/chrono/stopwatches/stopwatch_scoped.hpp
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/chrono/stopwatches/stopwatch.hpp
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/chrono/stopwatches/simple_stopwatch.hpp

... a lot more of that ...

common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/fusion/mpl/pop_front.hpp
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/fusion/mpl/pop_back.hpp
common.copy /home/simon/Projects/LinuxCompile/target-64-sse2/include/boost/fusion/mpl/insert_range.hpp
gcc.compile.c++ bin.v2/libs/locale/build/gcc-7.5.0/release/link-static/threading-multi/std/std_backend.o
gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/list.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/list.hpp:8,
                 from libs/python/src/list.cpp:5:
./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
 # include <pyconfig.h>
           ^~~~~~~~~~~~
compilation terminated.

    "g++"   -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/local/include/python3.7" -c -o "bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/list.o" "libs/python/src/list.cpp"

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/list.o...
gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/dict.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/dict.hpp:8,
                 from libs/python/src/dict.cpp:4:
./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
 # include <pyconfig.h>
           ^~~~~~~~~~~~
compilation terminated.

    "g++"   -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/local/include/python3.7" -c -o "bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/dict.o" "libs/python/src/dict.cpp"

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/dict.o...
gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/long.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/long.hpp:8,
                 from libs/python/src/long.cpp:5:
./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
 # include <pyconfig.h>
           ^~~~~~~~~~~~
compilation terminated.

    "g++"   -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/local/include/python3.7" -c -o "bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/long.o" "libs/python/src/long.cpp"

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/long.o...
gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/tuple.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/tuple.hpp:8,
                 from libs/python/src/tuple.cpp:5:
./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
 # include <pyconfig.h>
           ^~~~~~~~~~~~
compilation terminated.

    "g++"   -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/local/include/python3.7" -c -o "bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/tuple.o" "libs/python/src/tuple.cpp"

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/tuple.o...
gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/object_protocol.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/object_protocol.hpp:8,
                 from libs/python/src/object_protocol.cpp:6:
./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
 # include <pyconfig.h>
           ^~~~~~~~~~~~
compilation terminated.

    "g++"   -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/local/include/python3.7" -c -o "bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/object_protocol.o" "libs/python/src/object_protocol.cpp"

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/object_protocol.o...
gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/module.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/scope.hpp:8,
                 from libs/python/src/module.cpp:9:
./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
 # include <pyconfig.h>
           ^~~~~~~~~~~~
compilation terminated.

    "g++"   -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/local/include/python3.7" -c -o "bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/module.o" "libs/python/src/module.cpp"

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/module.o...
gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/import.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/ssize_t.hpp:9,
                 from ./boost/python/object.hpp:8,
                 from ./boost/python/import.hpp:8,
                 from libs/python/src/import.cpp:6:
./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
 # include <pyconfig.h>
           ^~~~~~~~~~~~
compilation terminated.

    "g++"   -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -w -mtune=generic -mno-ssse3 -std=gnu++98 -mmmx -msse -msse2 -O2 -pipe -mfpmath=sse -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DBOOST_PYTHON_STATIC_LIB -DNDEBUG  -I"." -I"/usr/local/include/python3.7" -c -o "bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/import.o" "libs/python/src/import.cpp"

...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-7.5.0/release/link-static/threading-multi/import.o...
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/convert.o
gcc.compile.c++ bin.v2/libs/locale/build/gcc-7.5.0/release/link-static/threading-multi/shared/localization_backend.o
gcc.compile.c++ bin.v2/libs/locale/build/gcc-7.5.0/release/link-static/threading-multi/shared/mo_lambda.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/config_file.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/winmain.o
gcc.compile.c++ bin.v2/libs/locale/build/gcc-7.5.0/release/link-static/threading-multi/shared/message.o
gcc.compile.c++ bin.v2/libs/locale/build/gcc-7.5.0/release/link-static/threading-multi/posix/numeric.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/split.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/variables_map.o
gcc.compile.c++ bin.v2/libs/locale/build/gcc-7.5.0/release/link-static/threading-multi/std/numeric.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/options_description.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/cmdline.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/value_semantic.o
gcc.compile.c++ bin.v2/libs/program_options/build/gcc-7.5.0/release/link-static/threading-multi/parsers.o
...failed updating 7 targets...
...updated 6826 targets...
 !!! Boost compilation failed
Dade916 commented 4 years ago

AFAIK I need the same version as the Blender I'm building for, so 3.7.

No, not on Linux, you can use any v3.x version.

./boost/python/detail/wrap_python.hpp:50:11: fatal error: pyconfig.h: No such file or directory
# include <pyconfig.h>

I guess you have not installed "python3-dev" (i.e "sudo apt-get install python3-dev").

Theverat commented 4 years ago

I guess you have not installed "python3-dev" (i.e "sudo apt-get install python3-dev").

It was indeed not installed. However, after the installation I still get exactly the same errors. The build script output says: * Using Python include path: /usr/include/python3.6m But when I look in /usr/include/python3.6m/, the file pyconfig.h is there.

Dade916 commented 4 years ago

There may be a mess on your system:

 * Using Python library path: /home/simon/Projects/LinuxCompile/target-64-sse2/lib/libpython3.7.a
 * Using Python include path: /usr/include/python3.6m

You are mixing Python v3.6 and v3.7. Try to start from a clean LinuxCompile directory too, if you have tried to compile with both versions. Or, least delete "LinuxCompile/target-64-sse2" directory.

Theverat commented 4 years ago

Ok, after taking a break and going at this with a clear mind and a fresh Ubuntu 19.10 on a live USB, I got a small script that works. I have pasted it to the wiki page: http://wiki.luxcorerender.org/Compiling_LuxCore#Static_link_environment And I'll add it to this repo as well, so users will be able to compile Lux on Linux with one command.