search

facebookresearch / CompilerGym

Reinforcement learning environments for compiler and program optimization tasks
https://compilergym.ai/
MIT License
879 stars 123 forks source link

Prototype modern CMake standalone build #795

Open ChrisCummins opened 1 year ago

ChrisCummins commented 1 year ago

Tracking issue for an overhauled and simplified build that targets only CMake.

Current working branch for this is: https://github.com/ChrisCummins/CompilerGym/tree/v0.3.0 🚨 Not ready for use 🚨

I have a working build for the LLVM service, but it is not hemetic:

$ ldd /dev/shm/cummins/compiler_gym/build/llvm/llvm_env_service
        ...
        libgflags.so.2.2 => /lib/x86_64-linux-gnu/libgflags.so.2.2 (0x00007fc1825a4000)
        libglog.so.0 => /lib/x86_64-linux-gnu/libglog.so.0 (0x00007fc182571000)
        libprotobuf.so.31 => /private/home/cummins/.conda/envs/compiler_gym/lib/libprotobuf.so.31 (0x00007fc1822e1000)

Next steps are to add gflags and glog to conda deps, and figure out how to ship the required libraries from conda libs, or link statically.

cgao3 commented 1 year ago

I tried this branch.. again I failed at last step.. with following message like 

Subprocess.cc
'bash' '-c' '"/home/cgao3/CompilerGym/compiler_gym/make_config.py" --out-file-path "/home/cgao3/cg_build/compiler_gym/config.py"  --enable-llvm-env'
'bash' '-c' 'cd "/home/cgao3/CompilerGym/compiler_gym/.." && "/home/cgao3/anaconda3/envs/compiler_gym/bin/python3.8" setup.py --build-dir "/home/cgao3/cg_build/compiler_gym/../py_pkg/build" bdist_wheel --package-dir "../cg_build/compiler_gym/.." --dist-dir "/home/cgao3/cg_build/compiler_gym/../py_pkg/dist" --bdist-dir "/home/cgao3/cg_build/compiler_gym/../py_pkg/bdist" --get-wheel-filename'
-- Configuring done
CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    ObservationSpaces.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:266 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    Observation.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:237 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    LlvmSession.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:190 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    Benchmark.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:44 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    ActionSpace.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:25 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  Cannot find source file:

    RunService.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:14 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    BenchmarkDynamicConfig.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:70 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    BenchmarkFactory.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:86 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  Cannot find source file:

    ComputeObservation.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:115 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  Cannot find source file:

    LlvmExtractOne.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:173 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    Cost.cc
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:144 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    CompilationSession.cc
Call Stack (most recent call first):
  compiler_gym/service/CMakeLists.txt:35 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    Proto.cc
Call Stack (most recent call first):
  compiler_gym/service/proto/CMakeLists.txt:67 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    BenchmarkCache.cc
Call Stack (most recent call first):
  compiler_gym/service/runtime/CMakeLists.txt:38 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    CreateAndRunCompilerGymServiceImpl.cc
Call Stack (most recent call first):
  compiler_gym/service/runtime/CMakeLists.txt:111 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    LLLexer.cc
Call Stack (most recent call first):
  compiler_gym/third_party/LexedIr/CMakeLists.txt:11 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  Cannot find source file:

    compute_autophase.cc
Call Stack (most recent call first):
  compiler_gym/third_party/autophase/CMakeLists.txt:41 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    InstCount.cc
Call Stack (most recent call first):
  compiler_gym/third_party/autophase/CMakeLists.txt:21 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    InstCount.cc
Call Stack (most recent call first):
  compiler_gym/third_party/llvm/CMakeLists.txt:51 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    RunfilesPath.cc
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:67 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  Cannot find source file:

    Subprocess.cc
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:88 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:158 (add_library):
  Cannot find source file:

    Runtime.h
Call Stack (most recent call first):
  compiler_gym/service/runtime/CMakeLists.txt:18 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:158 (add_library):
  Cannot find source file:

    CompilerGymService.h
Call Stack (most recent call first):
  compiler_gym/service/runtime/CMakeLists.txt:67 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:158 (add_library):
  Cannot find source file:

    CompilerGymServiceImpl.h
Call Stack (most recent call first):
  compiler_gym/service/runtime/CMakeLists.txt:85 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:158 (add_library):
  Cannot find source file:

    GrpcStatusMacros.h
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:54 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:158 (add_library):
  Cannot find source file:

    EnumUtil.h
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:42 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:158 (add_library):
  Cannot find source file:

    StrLenConstexpr.h
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:80 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:158 (add_library):
  Cannot find source file:

    Unreachable.h
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:105 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target:
  compiler_gym__envs__llvm__service__ObservationSpaces
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:266 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__envs__llvm__service__Observation
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:237 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__envs__llvm__service__LlvmSession
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:190 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__envs__llvm__service__Benchmark
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:44 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__envs__llvm__service__ActionSpace
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:25 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  No SOURCES given to target:
  compiler_gym__envs__llvm__service__compiler_gym-llvm-service
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:14 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target:
  compiler_gym__envs__llvm__service__BenchmarkDynamicConfig
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:70 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target:
  compiler_gym__envs__llvm__service__BenchmarkFactory
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:86 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  No SOURCES given to target:
  compiler_gym__envs__llvm__service__compute_observation
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:115 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  No SOURCES given to target:
  compiler_gym__envs__llvm__service__llvm-extract-one
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:173 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__envs__llvm__service__Cost
Call Stack (most recent call first):
  compiler_gym/envs/llvm/service/CMakeLists.txt:144 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__service__CompilationSession
Call Stack (most recent call first):
  compiler_gym/service/CMakeLists.txt:35 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__service__proto__proto_cc
Call Stack (most recent call first):
  compiler_gym/service/proto/CMakeLists.txt:67 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__service__runtime__BenchmarkCache
Call Stack (most recent call first):
  compiler_gym/service/runtime/CMakeLists.txt:38 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target:
  compiler_gym__service__runtime__CreateAndRunCompilerGymServiceImpl
Call Stack (most recent call first):
  compiler_gym/service/runtime/CMakeLists.txt:111 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__third_party__LexedIr__LexedIr
Call Stack (most recent call first):
  compiler_gym/third_party/LexedIr/CMakeLists.txt:11 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_binary.cmake:72 (add_executable):
  No SOURCES given to target:
  compiler_gym__third_party__autophase__compute_autophase
Call Stack (most recent call first):
  compiler_gym/third_party/autophase/CMakeLists.txt:41 (cg_cc_binary)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__third_party__autophase__InstCount
Call Stack (most recent call first):
  compiler_gym/third_party/autophase/CMakeLists.txt:21 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__third_party__llvm__InstCount
Call Stack (most recent call first):
  compiler_gym/third_party/llvm/CMakeLists.txt:51 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__util__RunfilesPath
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:67 (cg_cc_library)

CMake Error at build_tools/cmake/cg_cc_library.cmake:99 (add_library):
  No SOURCES given to target: compiler_gym__util__Subprocess
Call Stack (most recent call first):
  compiler_gym/util/CMakeLists.txt:88 (cg_cc_library)

CMake Generate step failed.  Build files cannot be regenerated correctly.
ChrisCummins commented 1 year ago

Oh yeah sorry I should make it clear that this is nowhere yet ready for use and is broken in many many ways 🙂