Error when running ./build-scalehls.sh

Fiwo735 commented 2 years ago

Following the steps mentioned in the README, I cloned the repo, cd into it, and tried to build it using ./build-scalehls.sh. I got the following error when trying with Ninja installed:

Synchronizing submodule url for 'polygeist'
>>> Unified MLIR, Clang, and ScaleHLS build...
ninja: error: loading 'build.ninja': No such file or directory

I then disabled Ninja to try to run with make and I got a similar error:

Synchronizing submodule url for 'polygeist'
>>> Unified MLIR, Clang, and ScaleHLS build...
makeL *** No targets specified and no makefile found. Stop.

I use Conda to manage my environment and I have the following library versions:

Python: 3.6.7
CMake: 3.23.2
Ninja: 1.11.0
Clang: 3.4.2
LLD: 14.0.4

It is likely some silly error on my side, but given I run into issues on the literal first step of the guide, I am a bit lost on what to try or change, so I would be very happy to get some help here, thanks!

hanchenye commented 2 years ago

Hi @Fiwo735, I'm not sure what is happening here. Could you try to manually create the build folder and config cmake following the commands in the script?

hanchenye commented 2 years ago

You should be able to start from here: https://github.com/hanchenye/scalehls/blob/0f9627a9f203de26c793888c195f262b83847e0f/build-scalehls.sh#L34-L61

Fiwo735 commented 2 years ago

Hi @hanchenye, thank you very much for your quick response! Below is the output when trying to manually build:

(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls:~$ mkdir -p build
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls:~$ cd build
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/build:~$ SCALEHLS_DIR=/mnt/ccnas2/bdp/faw18/FYP/scalehls/
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/build:~$ PYBIND=OFF
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/build:~$ JOBS=1
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/build:~$ CMAKE_GENERATOR="Unix Makefiles"
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/build:~$ cmake -G "${CMAKE_GENERATOR}" ../polygeist/llvm-project/llvm -DLLVM_ENABLE_PROJECTS="mlir;clang" -DLLVM_EXTERNAL_PROJECTS="scalehls" -DLLVM_EXTERNAL_SCALEHLS_SOURCE_DIR="${SCALEHLS_DIR}" -DLLVM_TARGETS_TO_BUILD="host" -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=DEBUG -DMLIR_ENABLE_BINDINGS_PYTHON="${PYBIND:=OFF}" -DSCALEHLS_ENABLE_BINDINGS_PYTHON="${PYBIND:=OFF}" -DLLVM_PARALLEL_LINK_JOBS="${JOBS:=}" -DLLVM_USE_LINKER=lld -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
-- The C compiler identification is Clang 14.0.4
-- The CXX compiler identification is Clang 3.4.2
-- The ASM compiler identification is Clang
-- Found assembler: /mnt/ccnas2/bdp/faw18/Anaconda3/envs/scale_hls/bin/clang
-- Check for working C compiler: /mnt/ccnas2/bdp/faw18/Anaconda3/envs/scale_hls/bin/clang
-- Check for working C compiler: /mnt/ccnas2/bdp/faw18/Anaconda3/envs/scale_hls/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/clang++
-- Check for working CXX compiler: /usr/bin/clang++ -- broken
CMake Error at /mnt/ccnas2/bdp/faw18/Anaconda3/envs/scale_hls/share/cmake-3.16/Modules/CMakeTestCXXCompiler.cmake:53 (message):
  The C++ compiler

    "/usr/bin/clang++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /mnt/ccnas2/bdp/faw18/FYP/scalehls/build/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/gmake cmTC_c14de/fast && /usr/bin/gmake -f CMakeFiles/cmTC_c14de.dir/build.make CMakeFiles/cmTC_c14de.dir/build
    gmake[1]: Entering directory `/mnt/ccnas2/bdp/faw18/FYP/scalehls/build/CMakeFiles/CMakeTmp'
    Building CXX object CMakeFiles/cmTC_c14de.dir/testCXXCompiler.cxx.o
    /usr/bin/clang++    -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /mnt/ccnas2/bdp/faw18/Anaconda3/envs/scale_hls/include    -o CMakeFiles/cmTC_c14de.dir/testCXXCompiler.cxx.o -c /mnt/ccnas2/bdp/faw18/FYP/scalehls/build/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
    clang: error: unknown argument: '-fstack-protector-strong'
    clang: error: unknown argument: '-fno-plt'
    gmake[1]: *** [CMakeFiles/cmTC_c14de.dir/testCXXCompiler.cxx.o] Error 1
    gmake[1]: Leaving directory `/mnt/ccnas2/bdp/faw18/FYP/scalehls/build/CMakeFiles/CMakeTmp'
    gmake: *** [cmTC_c14de/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:49 (project)

-- Configuring incomplete, errors occurred!
See also "/mnt/ccnas2/bdp/faw18/FYP/scalehls/build/CMakeFiles/CMakeOutput.log".
See also "/mnt/ccnas2/bdp/faw18/FYP/scalehls/build/CMakeFiles/CMakeError.log".
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/build:~$ which clang
/mnt/ccnas2/bdp/faw18/Anaconda3/envs/scale_hls/bin/clang
(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/build:~$ clang --version
clang version 14.0.4 (https://github.com/conda-forge/clangdev-feedstock 20b07913b60d39303fd40b57fe29fad61650dad0)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /mnt/ccnas2/bdp/faw18/Anaconda3/envs/scale_hls/bin

Seeing that the error comes from unknown arguments to clang++, or more precisely /usr/bin/clang++ (which is likely outdated), I realized that my Conda environment hasn't installed a new version for clang++ and only for clang. That might be a rare case, but it might be useful to update the README with clang++/clangxx as a prerequisite, to prevent people using Conda from having a similar issue.

After adding clangxx as a requirement for my environment and cleaning the build/ directory, I was able to run ./build-scalehls.sh. After around 30 minutes of compilation, the end of the logs printed to terminal from building the project can be seen below:

[100%] Running the clang-to-mlir regression tests
                                                                                                                                                                      -- Testing: 143 tests, 72 workers --                                                                                                                                                                     
  0% [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------]
XPASS: mlir-clang :: Verification/gettimeofday.c (62 of 143)
******************** TEST 'mlir-clang :: Verification/gettimeofday.c' FAILED ********************
Script:
--
: 'RUN: at line 1';   /mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/build/bin/mlir-clang /mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/tools/mlir-clang/Test/Verification/gettimeofday.c -I /mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/llvm-project/llvm/../clang/lib/Headers -I /mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/tools/mlir-clang/Test/polybench/utilities --function=alloc -S | /mnt/ccnas2/bdp/faw18/FYP/scalehls/build/bin/FileCheck /mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/tools/mlir-clang/Test/Verification/gettimeofday.c
--
Exit Code: 0
********************
********************
Unexpectedly Passed Tests (1):
  mlir-clang :: Verification/gettimeofday.c

Testing Time: 7.25s
  Passed             : 136
  Expectedly Failed  :   6
  Unexpectedly Passed:   1
make[3]: *** [tools/mlir-clang/Test/CMakeFiles/check-mlir-clang.dir/build.make:58: tools/mlir-clang/Test/CMakeFiles/check-mlir-clang] Error 1
make[3]: Leaving directory '/mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/build'
make[2]: *** [CMakeFiles/Makefile2:977: tools/mlir-clang/Test/CMakeFiles/check-mlir-clang.dir/all] Error 2
make[2]: Leaving directory '/mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/build'
make[1]: *** [CMakeFiles/Makefile2:984: tools/mlir-clang/Test/CMakeFiles/check-mlir-clang.dir/rule] Error 2
make[1]: Leaving directory '/mnt/ccnas2/bdp/faw18/FYP/scalehls/polygeist/build'
make: *** [Makefile:476: check-mlir-clang] Error 2

I was able to sucessfully run the HLS C/C++ example polybench/gemm, however, when trying the Pytorch Model pytorch/resnet18 one, I got the following output:

(scale_hls) /mnt/ccnas2/bdp/faw18/FYP/scalehls/samples/pytorch/resnet18:~$ python3 export_resnet18_mlir.py | torch-mlir-opt -torchscript-module-to-torch-backend-pipeline="optimize=true" -torch-backend-to-tosa-backend-pipeline="optimize=true" > resnet18.mlir
bash: torch-mlir-opt: command not found
Traceback (most recent call last):
  File "/mnt/ccnas2/bdp/faw18/FYP/scalehls/samples/pytorch/resnet18/export_resnet18_mlir.py", line 5, in <module>
    from torch_mlir.dialects.torch.importer.jit_ir import ClassAnnotator, ModuleBuilder
ModuleNotFoundError: No module named 'torch_mlir'

Am I missing some more dependencies or was the build process not actually fully successul?

hanchenye commented 2 years ago

Hey, great to know that the script is working!

On many testing machines, including my own one, the gettimeofday.c test case is failing but apparently on your machine it passes. You can follow #51 and run build-scalehls.sh again to finish the process.

For the PyTorch models, torch-mlir is more like an upstream tool instead of a part of scalehls, so torch-mlir is not a submodule and not built by build-scalehls.sh. I'd suggest to use the hash in https://github.com/hanchenye/scalehls#compiling-pytorch-model to build torch-mlir first to have the samples in pytorch/resnet18 working.

xinqiaozhang commented 1 year ago

Hi,

I would like to follow up with this question, when I use the hash in github link to build torch-mlir and I always got the error saying "fatal error: 'ATen/native/quantized/cpu/packed_params.h' file not found" I am using Ubuntu 22.04, python 3.8.0. Could you please help? Thanks

UIUC-ChenLab / scalehls

Error when running ./build-scalehls.sh #50