mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.65k stars 1.64k forks source link

CMake TARGET_FILE Generator expression doesn't work for grpc subproject #10317

Open jboomer opened 2 years ago

jboomer commented 2 years ago

Describe the bug Trying to compile grpc as a cmake subproject. Unfortunately this fails during the step where it compiles some proto files itself. Particularly, this seems to happen because the TARGET_FILE expression which is supposed to resolve to the path of the protoc compiler and the grpc_cpp_plugin executable do not get filled in.

meson.build :

project('test-grpc-subproject', 'cpp')

cmake = import('cmake')
opt_var = cmake.subproject_options()
opt_var.add_cmake_defines({   'CMAKE_BUILD_TYPE' : 'Release'
                            , 'gRPC_INSTALL' : true
                            , 'gRPC_BUILD_GRPC_CSHARP_PLUGIN' : false
                            , 'gRPC_BUILD_GRPC_NODE_PLUGIN' : false
                            , 'gRPC_BUILD_GRPC_OBJECTIVE_C_PLUGIN' : false
                            , 'gRPC_BUILD_GRPC_PHP_PLUGIN' : false
                            , 'gRPC_BUILD_GRPC_RUBY_PLUGIN' : false
                            , 'gRPC_BUILD_TESTS' : false
                            , 'gRPC_SSL_PROVIDER' : 'package'})

grpc_sub_proj = cmake.subproject('grpc_cmake', options : opt_var)

message('CMaket targets:\n - ' + '\n - '.join(grpc_sub_proj.target_list()))

grpc_lib = grpc_sub_proj.dependency('grpc')
grpcpp_lib = grpc_sub_proj.dependency('grpc++')
protobuf_lib = grpc_sub_proj.dependency('libprotobuf')

subdir('src/grpc-example')

Failed build step:

FAILED: subprojects/grpc_cmake/channelz.grpc.pb.cc subprojects/grpc_cmake/channelz.grpc.pb.h subprojects/grpc_cmake/channelz_mock.grpc.pb.h subprojects/grpc_cmake/channelz.pb.cc subprojects/grpc_cmake/channelz.pb.h
/usr/bin/meson --internal cmake_run_ctgt -o subprojects/grpc_cmake/channelz.grpc.pb.cc subprojects/grpc_cmake/channelz.grpc.pb.h subprojects/grpc_cmake/channelz_mock.grpc.pb.h subprojects/grpc_cmake/channelz.pb.cc subprojects/grpc_cmake/channelz.pb.h -O /home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens/src/proto/grpc/channelz/channelz.grpc.pb.cc /home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens/src/proto/grpc/channelz/channelz.grpc.pb.h /home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens/src/proto/grpc/channelz/channelz_mock.grpc.pb.h /home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens/src/proto/grpc/channelz/channelz.pb.cc /home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens/src/proto/grpc/channelz/channelz.pb.h -d /home/jbo/git/meson-grpc-subproject/subprojects/grpc_cmake --grpc_out=generate_mock_code=true:/home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens --cpp_out=/home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens --plugin=protoc-gen-grpc= -I . -I /home/jbo/git/meson-grpc-subproject/subprojects/grpc_cmake/third_party/protobuf/src src/proto/grpc/channelz/channelz.proto ';;;'
usage: meson [-h] -d D -o O [O ...] [-O [O ...]] ...
meson: error: unrecognized arguments: --grpc_out=generate_mock_code=true:/home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens --cpp_out=/home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens --plugin=protoc-gen-grpc= -I

In the above step, the path to protoc seems to be missing as well as the path to the grpc_cpp_plugin. Highlighted below where they should be:

...
 -d /home/jbo/git/meson-grpc-subproject/subprojects/grpc_cmake <PATH_TO_PROTOC> --grpc_out=generate_mock_code=true:/home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens --cpp_out=/home/jbo/git/meson-grpc-subproject/build/subprojects/grpc_cmake/__CMake_build/gens --plugin=protoc-gen-grpc=<PATH_TO_CPP_PLUGIN> -I . -I /home/jbo/git/meson-grpc-subproject/subprojects/grpc_cmake/third_party/protobuf/src src/proto/grpc/channelz/channelz.proto 

In the protobuf.cmake and CMakeLists.txt these are generated like:

set(_gRPC_PROTOBUF_PROTOC_EXECUTABLE $<TARGET_FILE:protoc>)

and

set(_gRPC_CPP_PLUGIN $<TARGET_FILE:grpc_cpp_plugin>)

respectively.

To Reproduce Use the meson build above with an executable that uses the dependencies.

Expected behavior protoc and grpcc_cpp_plugin are found and grpc compiles

system parameters

WaldonChen commented 2 years ago

I have the similar issue with scotch subproject. Changing mesonbuild/cmake/generator.py Line 69 to return arg works for me.

jboomer commented 2 years ago

Thanks, but that change doesn't work as the protoc and grpc_cpp_plugin are built by grpc itself, so they are not in the path. From a comment in generator.py:

    Most generator expressions are simply ignored for
    simplicety, however some are required for some common
    use cases.

I guess this is a more advanced use case that is not really supported. I guess the way to use grpc in a meson project would be to compile it outside of the project and use pkg-config to link it.

eli-schwartz commented 2 years ago

It's also possible to port grpc to meson and submit it to the WrapDB for others to use. Then you're not at the mercy of the quality of Meson's imperfect heuristics while trying to translate foreign build systems, and as a side benefit, you do not need cmake to be installed when building.

That being said, our cmake expert may be able to improve those heuristics to get this use case working, in which case, great!

mensinda commented 2 years ago

The main problem here is that protoc uses an executable that was not imported from a dependency and is declared inside the project. So basically:

add_executable(grpc_cpp_plugin
  src/compiler/cpp_plugin.cc
)

add_custom_command(
    OUTPUT "${_gRPC_PROTO_GENS_DIR}/${RELFIL_WE}.grpc.pb.cc"
           # more stuff...
    COMMAND ${_gRPC_PROTOBUF_PROTOC_EXECUTABLE}
    ARGS --plugin=protoc-gen-grpc=${_gRPC_CPP_PLUGIN}
          # more stuff...
    DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/${FILE_LOCATION} ${ABS_FIL} ${_gRPC_PROTOBUF_PROTOC} grpc_cpp_plugin
   # more stuff
)

I think I found a valid use case for exe.full_path() :) Now I just have to figure out how to teach the CMake backend how to make it work...


@WaldonChen, did you open an issue about that. Your fix is more a hack and likely only works by accident.

jboomer commented 2 years ago

@eli-schwartz : I'm neither an expert in cmake nor meson and that project looks positively daunting :) . Just evaluating solutions for now, compiling it outside the project would be ok although if it can work as a cmake subproject that would be awesome. From what I could see so far the mentioned issue is really the only roadblock, the rest seems to compile fine.

mensinda commented 2 years ago

@jboomer Implementing the missing feature in the CMake subproject is possible on the Meson side but might take a bit because it is not really straightforward.

WaldonChen commented 2 years ago

I opened an issue about my hack: https://github.com/mesonbuild/meson/issues/10328