search

leaningtech / cheerp-meta

Cheerp - a C/C++ compiler for Web applications - compiles to WebAssembly and JavaScript
https://labs.leaningtech.com/cheerp
Other
1.02k stars 50 forks source link

Bad https://docs.leaningtech.com/cheerp/Linux-build-instructions #147

Closed mingodad closed 1 year ago

mingodad commented 1 year ago

When trying to build cheerp-3.0 on a Ubuntu 18.04 64bits I'm getting an error when trying to build cheerp-libs/webgles/webgles.cpp that says it can't find #include <string> and looking at https://launchpad.net/~leaningtech-dev/+archive/ubuntu/cheerp-ppa I can see that there is cheerp-libcxx and cheerp-libcxx-libcxxabi but in the instructions to build from source there is only -DLLVM_ENABLE_PROJECTS=clang and probably it should be -DLLVM_ENABLE_PROJECTS=clang,libcxx,libcxxabi to also build and install the correspond libcxx to be able to resolve #include <string> .

alexp-sssup commented 1 year ago

Please make sure to follow all the build steps, libcxx and libcxxabi are built after musl here:

https://docs.leaningtech.com/cheerp/Linux-build-instructions#build-cheerp2-utilities-and-libraries-stable-version

mingodad commented 1 year ago

I don't see any mention to libcxx/libcxxabi there.

Are you sure about it ?

Here is my shell script adapted from that page to install on a local folder:

#!/bin/sh
MYCHEERP_PRFIX=$HOME/local/cheerp
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-compiler
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-utils
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-musl
git clone --depth 1 --branch cheerp-3.0 https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
sed -i '13a\  llvm-ar' llvm/CheerpCmakeConf.cmake
mycmake -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PRFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=all -G Ninja
ninja -C build -j4
ninja -C build install
cd ..

cd cheerp-utils
mycmake -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PRFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PRFIX/bin/llvm-ar s" AR="$MYCHEERP_PRFIX/bin/llvm-ar"  CC="$MYCHEERP_PRFIX/bin/clang -target cheerp" LD="$MYCHEERP_PRFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PRFIX
make clean
make -j8
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PRFIX/bin/llvm-ar s" AR="$MYCHEERP_PRFIX/bin/llvm-ar"  CC="$MYCHEERP_PRFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PRFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PRFIX
make clean
make -j8
make install
cd ../..

cd cheerp-compiler
mycmake -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PRFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs
ninja -C build_runtimes_genericjs install

mycmake -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PRFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PRFIX CHEERP_PREFIX=$MYCHEERP_PRFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PRFIX CHEERP_PREFIX=$MYCHEERP_PRFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PRFIX CHEERP_PREFIX=$MYCHEERP_PRFIX
cd ..
alexp-sssup commented 1 year ago

The libraries are built as part of the runtime steps

mingodad commented 1 year ago

As you can see my copy/paste of the instructions and it fails:

/home/mingo/local/cheerp/bin/llvm-ar s lib/libc.a
../tools/install.sh -D -m 644 lib/Scrt1.o /home/mingo/local/cheerp/lib/asmjs/Scrt1.o
../tools/install.sh -D -m 644 lib/crt1.o /home/mingo/local/cheerp/lib/asmjs/crt1.o
../tools/install.sh -D -m 644 lib/crti.o /home/mingo/local/cheerp/lib/asmjs/crti.o
../tools/install.sh -D -m 644 lib/crtn.o /home/mingo/local/cheerp/lib/asmjs/crtn.o
../tools/install.sh -D -m 644 lib/rcrt1.o /home/mingo/local/cheerp/lib/asmjs/rcrt1.o
../tools/install.sh -D -m 644 lib/libc.a /home/mingo/local/cheerp/lib/asmjs/libc.a
../tools/install.sh -D -m 644 lib/libm.a /home/mingo/local/cheerp/lib/asmjs/libm.a
../tools/install.sh -D -m 644 lib/librt.a /home/mingo/local/cheerp/lib/asmjs/librt.a
../tools/install.sh -D -m 644 lib/libpthread.a /home/mingo/local/cheerp/lib/asmjs/libpthread.a
../tools/install.sh -D -m 644 lib/libcrypt.a /home/mingo/local/cheerp/lib/asmjs/libcrypt.a
../tools/install.sh -D -m 644 lib/libutil.a /home/mingo/local/cheerp/lib/asmjs/libutil.a
../tools/install.sh -D -m 644 lib/libxnet.a /home/mingo/local/cheerp/lib/asmjs/libxnet.a
../tools/install.sh -D -m 644 lib/libresolv.a /home/mingo/local/cheerp/lib/asmjs/libresolv.a
../tools/install.sh -D -m 644 lib/libdl.a /home/mingo/local/cheerp/lib/asmjs/libdl.a
../tools/install.sh -D -m 644 lib/libc.bc /home/mingo/local/cheerp/lib/asmjs/libc.bc
../tools/install.sh -D -m 644 lib/crt1.bc /home/mingo/local/cheerp/lib/asmjs/crt1.bc
../tools/install.sh -D -m 644 obj/include/bits/alltypes.h /home/mingo/local/cheerp/include/bits/alltypes.h
../tools/install.sh -D -m 644 obj/include/bits/syscall.h /home/mingo/local/cheerp/include/bits/syscall.h
loading initial cache file runtimes/CheerpCmakeConf.cmake
-- The ASM compiler identification is unknown
-- Didn't find assembler
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_C_COMPILER:

    /opt/cheerp/bin/clang

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.

CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_CXX_COMPILER:

    /opt/cheerp/bin/clang++

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.

CMake Error at CMakeLists.txt:3 (project):
  No CMAKE_ASM_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "ASM" or the CMake cache entry CMAKE_ASM_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.

-- Warning: Did not find file Compiler/-ASM
-- Configuring incomplete, errors occurred!
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeFiles/CMakeOutput.log".
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeFiles/CMakeError.log".
ninja: Entering directory `build_runtimes_genericjs'
ninja: error: loading 'build.ninja': No such file or directory
ninja: Entering directory `build_runtimes_genericjs'
ninja: error: loading 'build.ninja': No such file or directory
loading initial cache file runtimes/CheerpCmakeConf.cmake
-- The ASM compiler identification is unknown
-- Didn't find assembler
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_C_COMPILER:

    /opt/cheerp/bin/clang

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.

CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_CXX_COMPILER:

    /opt/cheerp/bin/clang++

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.

CMake Error at CMakeLists.txt:3 (project):
  No CMAKE_ASM_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "ASM" or the CMake cache entry CMAKE_ASM_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.

-- Warning: Did not find file Compiler/-ASM
-- Configuring incomplete, errors occurred!
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeFiles/CMakeOutput.log".
See also "/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeFiles/CMakeError.log".
ninja: Entering directory `build_runtimes_wasm'
ninja: error: loading 'build.ninja': No such file or directory
ninja: Entering directory `build_runtimes_wasm'
ninja: error: loading 'build.ninja': No such file or directory
make: Entering directory '/tmp/cheerp/cheerp-libs/webgles'
mkdir -p /home/mingo/local/cheerp/include/GLES2/
cp -v webgles.h gl2.h /home/mingo/local/cheerp/include/GLES2/
'webgles.h' -> '/home/mingo/local/cheerp/include/GLES2/webgles.h'
'gl2.h' -> '/home/mingo/local/cheerp/include/GLES2/gl2.h'
/home/mingo/local/cheerp/bin/clang++  -I/home/mingo/local/clang-15/include -I/home/mingo/local/include -std=c++1y -c webgles.cpp -o webgles.bc -target cheerp -O3
In file included from webgles.cpp:12:
In file included from ./webgles.h:15:
In file included from /home/mingo/local/cheerp/bin/../include/client/cheerp/client.h:15:
/home/mingo/local/cheerp/bin/../include/client/cheerp/types.h:16:10: fatal error: 'string' file not found
#include <string>
         ^~~~~~~~
1 error generated.
Makefile:12: recipe for target 'webgles.bc' failed
make: *** [webgles.bc] Error 1
make: Leaving directory '/tmp/cheerp/cheerp-libs/webgles'
make: Entering directory '/tmp/cheerp/cheerp-libs/wasm'
mkdir -p /home/mingo/local/cheerp/lib/
cp -v libwasm.bc /home/mingo/local/cheerp/lib/
'libwasm.bc' -> '/home/mingo/local/cheerp/lib/libwasm.bc'
make: Leaving directory '/tmp/cheerp/cheerp-libs/wasm'
make: Entering directory '/tmp/cheerp/cheerp-libs/stdlibs'
make: *** No rule to make target '/home/mingo/local/cheerp/lib/genericjs/libc++.bc', needed by 'genericjs/libstdlibs.full.bc'.  Stop.
make: Leaving directory '/tmp/cheerp/cheerp-libs/stdlibs'
alexp-sssup commented 1 year ago 
CMake Error at CMakeLists.txt:3 (project):
  The CMAKE_C_COMPILER:

    /opt/cheerp/bin/clang
mingodad commented 1 year ago

It seems that there is /opt/cheerp/* hardcoded in several places and that prevents install on a user defined location. Would be nice to have all of then replaced by a macro/variable.

alexp-sssup commented 1 year ago

Passing an explicit -DCMAKE_INSTALL_PREFIX=... to the runtimes build step should work.

It is much easier for us to debug problems if users have the same layout as we do though.

mingodad commented 1 year ago

Passing an explicit -DCMAKE_INSTALL_PREFIX=... to the runtimes build step should work.

It is much easier for us to debug problems if users have the same layout as we do though.

I'm trying it now. Can that be added to the instructions ? Because some calls to cmake have it and some don't.

Here is the main offenders:

/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:20:CHEERP_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:67:CMAKE_CXX_COMPILER:STRING=/opt/cheerp/bin/clang++
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:90:CMAKE_C_COMPILER:STRING=/opt/cheerp/bin/clang
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:137:CMAKE_INSTALL_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:139:CMAKE_LINKER:STRING=/opt/cheerp/bin/llvm-link
/tmp/cheerp/cheerp-compiler/build_runtimes_genericjs/CMakeCache.txt:164:CMAKE_MODULE_PATH:STRING=/opt/cheerp/share/cmake/Modules
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:20:CHEERP_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:67:CMAKE_CXX_COMPILER:STRING=/opt/cheerp/bin/clang++
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:90:CMAKE_C_COMPILER:STRING=/opt/cheerp/bin/clang
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:137:CMAKE_INSTALL_PREFIX:STRING=/opt/cheerp
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:139:CMAKE_LINKER:STRING=/opt/cheerp/bin/llvm-link
/tmp/cheerp/cheerp-compiler/build_runtimes_wasm/CMakeCache.txt:164:CMAKE_MODULE_PATH:STRING=/opt/cheerp/share/cmake/Modules
alexp-sssup commented 1 year ago

I am happy to add that to the instruction if you can confirm that it works :-)

CMAKE_INSTALL_PREFIX should be enough, other definitions are derived from it

mingodad commented 1 year ago

Thanks ! It'll take around an hour to finish the build and if everything is OK I'll tell you.

mingodad commented 1 year ago

Also the number of threads to use are hardcoded and they don't match:

cd cheerp-compiler
...
ninja -C build -j4
...
cd cheerp-musl
...
make -j8
...
cd build_asmjs
...
make -j8

Probably removing the -j4 from ninja -C build -j4 and using make -j `nproc`.

mingodad commented 1 year ago

It doesn't build !

I've tried to build with g++-9.4 and with clang-15, none of then build, I'll look in more detail later.

Bellow I'm attaching the output of the build.

build.log.zip

alexp-sssup commented 1 year ago

The problem is that both the genericjs and the wasm version of the runtimes must be compiled before any of them are installed.

I have fixed README.md file to reflect this:

https://github.com/leaningtech/cheerp-compiler/blob/master/README.md#build-libcxx-and-libcxxabi

mingodad commented 1 year ago

And what about the info here https://docs.leaningtech.com/cheerp/Linux-build-instructions ?

alexp-sssup commented 1 year ago

Please follow the README, we will update the instructions on the website later

mingodad commented 1 year ago

Here is again the output of my attempt to build cheerp after changing the order pointed here https://github.com/leaningtech/cheerp-meta/issues/147#issuecomment-1589803702 .

cheerp-failed-build.log.zip

alexp-sssup commented 1 year ago

Make sure to wipe out the installation directory and try again.

mingodad commented 1 year ago

Can you try the script attached that I'm using or create your own based on the build instructions and try build on fresh machine/vm ? Or even better can a github action be made to test it ?

mingodad commented 1 year ago

I also noticed that cheerp seems to be limited to 32bits:

cheerp-latest/bin/clang++ -x c /dev/null -dM -E

define _ILP32 1

define ASMJS 1

define __ATOMIC_ACQUIRE 2

...

define __SIZEOF_LONG__ 4

define __SIZEOF_POINTER__ 4

define SIZEOF_PTRDIFF_T 4

define __SIZEOF_SHORT__ 2

define SIZEOF_SIZE_T 4

define SIZEOF_WCHAR_T 4

...

define __SIZE_MAX__ 4294967295U

define __SIZE_TYPE__ unsigned int

define __SIZE_WIDTH__ 32

alexp-sssup commented 1 year ago

We build every commit on CI and PPA, beside the order of installation corrected above they will work.

The installation directly must be clean, this is of course the case when building on CI.

The Web platform is considered to be 32-bit. There is some work on 64-bit wasm but it's not mature and will have significant performance issues.

mingodad commented 1 year ago

Why on https://github.com/leaningtech/cheerp-meta/issues/147#issuecomment-1589803702 it indicates to build https://github.com/leaningtech/cheerp-newlib (that is archived) but in https://docs.leaningtech.com/cheerp/Linux-build-instructions it indicated to build https://github.com/leaningtech/cheerp-musl ?

Which one is the one ?

alexp-sssup commented 1 year ago

Only musl, we need to review the docs, but the runtime steps are correct.

mingodad commented 1 year ago

I was looking at https://github.com/leaningtech/cheerp-compiler/blob/master/.circleci/config.yml and could see that you only build with debian/rpm but not the naked build described in https://docs.leaningtech.com/cheerp/Linux-build-instructions.

Can the naked build described in https://docs.leaningtech.com/cheerp/Linux-build-instructions be added to the circleci to allways check that it's working ?

alexp-sssup commented 1 year ago

We consider the Debian build to be our reference build.

The problems you are observing are due to the README being out-of-sync with the Debian instruction. Having another copy of the instructions would not help.

mingodad commented 1 year ago

It does help because in doubt I could see a working recipe on https://github.com/leaningtech/cheerp-compiler/blob/master/.circleci/config.yml .

alexp-sssup commented 1 year ago

You can see the recipe in the debian packages. It is equivalent.

I expect the build will work if you start with a fresh build directory, using musl and fixing the build order for the runtimes. Please do so, after you confirm the build work I will update the docs

mingodad commented 1 year ago

That was what I did and the result is there https://github.com/leaningtech/cheerp-meta/files/11743172/cheerp-failed-build.log.zip .

alexp-sssup commented 1 year ago

Again, you need to completely remove the installation directory before.

mingodad commented 1 year ago

And that was what I did !

alexp-sssup commented 1 year ago

I can see from the log that it is not the case.

In this log the runtimes fails at the genericjs step, which is the first one. This happens because a version is already installed in the destination directory, hence you have not removed the destination directory as requested.

In the original log the runtimes failed at the wasm step, which is the second one. This happened because the genericjs version was installed just before.

The build system of libc++ is broken (in upstream LLVM we believe) and cannot deal with the libraries being already installed in the system.

The solution to this problem is to install both the genericjs and wasm versions after both are built. This can only work in a fresh installation directory.

mingodad commented 1 year ago

Obs.: edited to remove my bad interpretation of the cmake debian recipe. Obs.: edited again with an cat $0 to log the entire build proccess and restarted it again.

I'm redoing it now with rm -R * on top of my script:

#!/bin/sh
cat $0
export PATH=$HOME/local/clang-16/bin:$PATH
export CC=clang
export CXX=clang++

MYCHEERP_PREFIX=$HOME/local/cheerp-latest
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD

rm -R $MYCHEERP_PREFIX
rm -R cheerp-compiler
rm -R cheerp-utils
rm -R cheerp-musl
rm -R cheerp-libs

git clone --depth 1  https://github.com/leaningtech/cheerp-compiler
git clone --depth 1  https://github.com/leaningtech/cheerp-utils
git clone --depth 1  https://github.com/leaningtech/cheerp-musl
git clone --depth 1  https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
mycmake -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build
ninja -C build install
cd ..

cd cheerp-utils
mycmake -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ../..

cd cheerp-compiler
mycmake  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs

mycmake  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm

ninja -C build_runtimes_genericjs install
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
cd ..
alexp-sssup commented 1 year ago

I'm redoing it now with rm -R * on top of my script:

Just to clarify, it's the installation directory that needs to be cleaned. Not the build directory.

mingodad commented 1 year ago

Just to not have any doubt in future conversations after I publish the build.log !

mingodad commented 1 year ago

Somehow it seems I've been missing cleaning out before build, see bellow the output of:

/usr/bin/time sh mk-it-last.sh > build.log 2>&1

buildcheerp-latest.log.zip

But now with the fresh cheerp-latest build and trying the hello.cpp I'm getting this error:

// The cheerp/clientlib.h header contains declarations for the browser APIs
#include <cheerp/clientlib.h>

// webMain is the entry point for web applications written in Cheerp
void webMain()
{
        client::console.log("Hello, World Wide Web!");
}

Running:

cheerp-env clang++ -target cheerp hello.cpp -o hello.js
/home/mingo/local/cheerp-latest/bin/llvm-link: No such file or directory
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
mingodad commented 1 year ago

Output with -v:

cheerp-env clang++ -v -target cheerp hello.cpp -o hello.js
clang version 16.0.0 (https://github.com/leaningtech/cheerp-compiler 528d56485decf17df06231e136bafb149eac300f)
Target: cheerp-leaningtech-webbrowser-genericjs
Thread model: posix
InstalledDir: /home/mingo/local/cheerp-latest/bin
 "/home/mingo/local/cheerp-latest/bin/clang-16" -cc1 -triple cheerp-leaningtech-webbrowser-genericjs -emit-llvm-bc -emit-llvm-uselists -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.cpp -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -fno-verbose-asm -no-integrated-as -mconstructor-aliases -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -fno-dwarf-directory-asm -v -fcoverage-compilation-dir=/home/mingo/dev/cheerp -resource-dir /home/mingo/local/cheerp-latest/lib/clang/16 -internal-isystem /home/mingo/local/cheerp-latest/bin/../include/c++/v1 -internal-isystem /home/mingo/local/cheerp-latest/lib/clang/16/include -internal-externc-isystem /home/mingo/local/cheerp-latest/bin/../include -internal-externc-isystem /home/mingo/local/cheerp-latest/bin/../include/client -no-opaque-pointers -fdeprecated-macro -fdebug-compilation-dir=/home/mingo/dev/cheerp -ferror-limit 19 -fno-rtti -fgnuc-version=4.2.1 -fno-threadsafe-statics -fcolor-diagnostics -o /tmp/hello-03b807.bc -x c++ hello.cpp
clang -cc1 version 16.0.0 based upon LLVM 16.0.0git default target cheerp-leaningtech-webbrowser-wasm
#include "..." search starts here:
#include <...> search starts here:
 /home/mingo/local/cheerp-latest/bin/../include/c++/v1
 /home/mingo/local/cheerp-latest/lib/clang/16/include
 /home/mingo/local/cheerp-latest/bin/../include
 /home/mingo/local/cheerp-latest/bin/../include/client
End of search list.
 "/home/mingo/local/cheerp-latest/bin/llvm-link" -o /tmp/hello-91c042.bc /tmp/hello-03b807.bc /home/mingo/local/cheerp-latest/bin/../lib/genericjs/libstdlibs.bc libsystem.bc
/home/mingo/local/cheerp-latest/bin/llvm-link: No such file or directory
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
alexp-sssup commented 1 year ago

"/home/mingo/local/cheerp-latest/bin/llvm-link" -o /tmp/hello-91c042.bc /tmp/hello-03b807.bc /home/mingo/local/cheerp-latest/bin/../lib/genericjs/libstdlibs.bc libsystem.bc

You are missing the libsystem.bc file, there is an additional step missing from the build doc. Please see the cheerp-libs README instead

https://github.com/leaningtech/cheerp-libs/#cheerp-syscall-library-installation-only-generic-js-version

mingodad commented 1 year ago

After adding this to my build script:

cd system
$cmake_cmd -B build_genericjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs
cd ..

It now build the hello.cpp.

But I also see that there are a lot more than in the build script:

make -C webgles install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C wasm install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_genericjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_asmjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
cd system
cmake -B build_genericjs -DCMAKE_INSTALL_PREFIX=/opt/cheerp -DCMAKE_TOOLCHAIN_FILE=/opt/cheerp/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs
cd ..
cmake -B build_asmjs -DCMAKE_INSTALL_PREFIX=/opt/cheerp -DCMAKE_TOOLCHAIN_FILE=/opt/cheerp/share/cmake/Modules/CheerpWasmToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_asmjs
make install -C build_asmjs
cd ..

Somehow they need be in the build script ?

And here is the full build script:

#!/bin/sh
cat $0
export PATH=$HOME/local/clang-16/bin:$PATH
export CC=clang
export CXX=clang++
cmake_cmd=mycmake
rmdir_cmd="rm -R"

MYCHEERP_PREFIX=$HOME/local/cheerp-latest
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD

$rmdir_cmd $MYCHEERP_PREFIX
$rmdir_cmd cheerp-compiler
$rmdir_cmd cheerp-utils
$rmdir_cmd cheerp-musl
$rmdir_cmd cheerp-libs

git clone --depth 1  https://github.com/leaningtech/cheerp-compiler
git clone --depth 1  https://github.com/leaningtech/cheerp-utils
git clone --depth 1  https://github.com/leaningtech/cheerp-musl
git clone --depth 1  https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
$cmake_cmd -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build
ninja -C build install
cd ..

cd cheerp-utils
$cmake_cmd -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ../..

cd cheerp-compiler
$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs

$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm

ninja -C build_runtimes_genericjs install
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
cd system
$cmake_cmd -B build_genericjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs
cd ..
cd ..
mingodad commented 1 year ago

Here is my latest build script that also include build_asmjs:

#!/bin/sh
cat $0
export PATH=$HOME/local/clang-16/bin:$PATH
export CC=clang
export CXX=clang++
cmake_cmd=mycmake
rmdir_cmd="rm -R"

MYCHEERP_PREFIX=$HOME/local/cheerp-latest
#mkdir cheerp
#cd cheerp
export CHEERP_SRC=$PWD

$rmdir_cmd $MYCHEERP_PREFIX
$rmdir_cmd cheerp-compiler
$rmdir_cmd cheerp-utils
$rmdir_cmd cheerp-musl
$rmdir_cmd cheerp-libs

git clone --depth 1  https://github.com/leaningtech/cheerp-compiler
git clone --depth 1  https://github.com/leaningtech/cheerp-utils
git clone --depth 1  https://github.com/leaningtech/cheerp-musl
git clone --depth 1  https://github.com/leaningtech/cheerp-libs

cd cheerp-compiler
$cmake_cmd -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build
ninja -C build install
cd ..

cd cheerp-utils
$cmake_cmd -B build -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX .
make -C build install
cd ..

cd cheerp-musl
mkdir build_genericjs
cd build_genericjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ..
mkdir build_asmjs
cd build_asmjs
RANLIB="$MYCHEERP_PREFIX/bin/llvm-ar s" AR="$MYCHEERP_PREFIX/bin/llvm-ar"  CC="$MYCHEERP_PREFIX/bin/clang -target cheerp-wasm" LD="$MYCHEERP_PREFIX/bin/llvm-link" CFLAGS="-Wno-int-conversion" ../configure --target=cheerp-wasm --disable-shared --prefix=$MYCHEERP_PREFIX
make clean
make -j `nproc`
make install
cd ../..

cd cheerp-compiler
$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs

$cmake_cmd  -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm

ninja -C build_runtimes_genericjs install
ninja -C build_runtimes_wasm install
cd ..

cd cheerp-libs
make -C webgles install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C wasm install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
make -C stdlibs install INSTALL_PREFIX=$MYCHEERP_PREFIX CHEERP_PREFIX=$MYCHEERP_PREFIX
cd system
$cmake_cmd -B build_genericjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_genericjs
make install -C build_genericjs

$cmake_cmd -B build_asmjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_asmjs
make install -C build_asmjs

$cmake_cmd -B build_asmjs -DCMAKE_INSTALL_PREFIX=$MYCHEERP_PREFIX -DCMAKE_TOOLCHAIN_FILE=$MYCHEERP_PREFIX/share/cmake/Modules/CheerpWasmToolchain.cmake -DCMAKE_BUILD_TYPE=Release .
make -C build_asmjs
make install -C build_asmjs
cd ../..
alexp-sssup commented 1 year ago 
make -C stdlibs install INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_genericjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp
make -C stdlibs install_asmjs INSTALL_PREFIX=/opt/cheerp CHEERP_PREFIX=/opt/cheerp

The install target will run install_genericjs and install_asmjs internally, there is no need to run them manually as well.

I have updated the READMEs and the docs website to fix the various issues

mingodad commented 1 year ago

At the end I was right that there is BAD instructions to build cheerp from sources, but you @alexp-sssup did not really checked it before instantly close this issue and we've lost a lot of time and energy to somehow have a working build script.

I hope you take the reported issues a bit more seriously, to not give a bad impression for this project !

mingodad commented 1 year ago

Looking at https://docs.leaningtech.com/cheerp/Linux-build-instructions I can see that you have updated the instructions to build latest.

But the build instructions to build stable remain broken.

mingodad commented 1 year ago

Also the install order is still not fixed:

cd cheerp-compiler
cmake -S runtimes -B build_runtimes_genericjs -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="/opt/cheerp/share/cmake/Modules/CheerpToolchain.cmake"
ninja -C build_runtimes_genericjs
ninja -C build_runtimes_genericjs install

cmake -S runtimes -B build_runtimes_wasm -GNinja -C runtimes/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="/opt/cheerp/share/cmake/Modules/CheerpWasmToolchain.cmake"
ninja -C build_runtimes_wasm
ninja -C build_runtimes_wasm install
cd ..

It seems that you are not aplying attention to the whole documentation scattered in several places.

alexp-sssup commented 1 year ago

Please refresh the page, you are most likely not seeing updated content.

You can see the source code here as confirmation: https://github.com/leaningtech/cheerp-meta/blob/master/pages/Linux-build-instructions.md

mingodad commented 1 year ago

Please publish a full working script (maybe the one that I've showed here and worked).

mingodad commented 1 year ago

Ok I can see here https://github.com/leaningtech/cheerp-meta/blob/master/pages/Linux-build-instructions.md the fixes.

Could you use a variable instead of repeating /opt/cheerp everywhere (even better a full working script like mine).

alexp-sssup commented 1 year ago

Please refresh the docs again, a CHEERP_DEST variable is now used.

mingodad commented 1 year ago

I just refreshed and it's a lot better !

But I still do not see -DCMAKE_INSTALL_PREFIX=${CHEERP_DEST} in :

cd cheerp-compiler
cmake -S llvm -B build -C llvm/CheerpCmakeConf.cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang -G Ninja
ninja -C build -j4
ninja -C build install
cd ..
mingodad commented 1 year ago

Please create a whole working script and add it to you circleci to always be sure that it works.

mingodad commented 1 year ago

Also run it yourself instead of imagining/guessing that it works !

alexp-sssup commented 1 year ago

The instructions should now be updated to consider the prefix also for the compiler step.

We do not plan to publish an additional build script since that defeats the purpose of following the build instructions.

If a user just wants to just use Cheerp, he could use the binary packages.

If a user builds we source we assume that he wants to understand the build process. A script to run blindly would not achieve that.

Thanks for helping us making the build instruction more clear.