Open dantesun opened 5 months ago
Hi Dante! Thanks for giving the tool a go. Sorry we got you snagged on this rough edge.
Here's my analysis:
@dantesun, could I ask you to try the following two things (without any sysroot workarounds you'd added to satisfy the tool):
EM_BIN_PATH
environment variable has been auto-set in these cc_libraries.If those don't work, please tag kon72 and ask him if he know's what's up. He might be able to help better, since having last touched that code and worked on emscripten support.
Cheers, Chris
@cpsauer Thanks for your quick response. I tried the two things you said but unfortunately my problem still exists. @kon72 Could you take a look at this issue, thanks.
I am using bazel 5.4.0.
BUILD
load("@hedron_compile_commands//:refresh_compile_commands.bzl", "refresh_compile_commands")
refresh_compile_commands(
name = "refresh_compile_commands",
targets = ["//extensions/...", "//common/..."]
)
WORKSPACE
workspace(name = "istio_ecosystem_wasm_extensions")
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
# Hedron's Compile Commands Extractor for Bazel
# https://github.com/hedronvision/bazel-compile-commands-extractor
http_archive(
name = "hedron_compile_commands",
# Replace the commit hash (0e990032f3c5a866e72615cf67e5ce22186dcb97) in both places (below) with the latest (https://github.com/hedronvision/bazel-compile-commands-extractor/commits/main), rather than using the stale one here.
# Even better, set up Renovate and let it do the work for you (see "Suggestion: Updates" in the README).
url = "https://github.com/hedronvision/bazel-compile-commands-extractor/archive/5bcb0bd8a917b2b48fb5dc55818515f4be3b63ff.tar.gz",
strip_prefix = "bazel-compile-commands-extractor-5bcb0bd8a917b2b48fb5dc55818515f4be3b63ff",
# When you first run this tool, it'll recommend a sha256 hash to put here with a message like: "DEBUG: Rule 'hedron_compile_commands' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = ..."
)
load("@hedron_compile_commands//:workspace_setup.bzl", "hedron_compile_commands_setup")
hedron_compile_commands_setup()
load("@hedron_compile_commands//:workspace_setup_transitive.bzl", "hedron_compile_commands_setup_transitive")
hedron_compile_commands_setup_transitive()
load("@hedron_compile_commands//:workspace_setup_transitive_transitive.bzl", "hedron_compile_commands_setup_transitive_transitive")
hedron_compile_commands_setup_transitive_transitive()
load("@hedron_compile_commands//:workspace_setup_transitive_transitive_transitive.bzl", "hedron_compile_commands_setup_transitive_transitive_transitive")
hedron_compile_commands_setup_transitive_transitive_transitive()
load("//bazel:third_party.bzl", "wasm_extension_dependency")
wasm_extension_dependency()
load(
"@io_bazel_rules_docker//repositories:repositories.bzl",
container_repositories = "repositories",
)
container_repositories()
load("@io_bazel_rules_docker//repositories:deps.bzl", container_deps = "deps")
container_deps()
PROXY_WASM_CPP_SDK_SHA = "fd0be8405db25de0264bdb78fae3a82668c03782"
PROXY_WASM_CPP_SDK_SHA256 = "c57de2425b5c61d7f630c5061e319b4557ae1f1c7526e5a51c33dc1299471b08"
http_archive(
name = "proxy_wasm_cpp_sdk",
sha256 = PROXY_WASM_CPP_SDK_SHA256,
strip_prefix = "proxy-wasm-cpp-sdk-" + PROXY_WASM_CPP_SDK_SHA,
url = "https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/archive/" + PROXY_WASM_CPP_SDK_SHA + ".tar.gz",
)
load("@proxy_wasm_cpp_sdk//bazel/dep:deps.bzl", "wasm_dependencies")
wasm_dependencies()
load("@proxy_wasm_cpp_sdk//bazel/dep:deps_extra.bzl", "wasm_dependencies_extra")
wasm_dependencies_extra()
load("@istio_ecosystem_wasm_extensions//bazel:wasm.bzl", "wasm_libraries")
wasm_libraries()
# To import proxy wasm cpp host, which will be used in unit testing.
load("@proxy_wasm_cpp_host//bazel:repositories.bzl", "proxy_wasm_cpp_host_repositories")
proxy_wasm_cpp_host_repositories()
load("@proxy_wasm_cpp_host//bazel:dependencies.bzl", "proxy_wasm_cpp_host_dependencies")
proxy_wasm_cpp_host_dependencies()
I got errors when running bazel run :refresh_compile_commands
INFO: Build completed successfully, 5 total actions [INFO 19:47:06.180 src/main/cpp/blaze_util_posix.cc:308] Invoking binary /bin/bash in /home/dsun/.cache/bazel/_bazel_dsun/c4e768d90cc77b1af937ef4d90fbb834/execroot/istio_ecosystem_wasm_extensions/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/istio_ecosystem_wasm_extensions >>> Analyzing commands used in //extensions/... [INFO 19:47:06.808 src/main/cpp/option_processor.cc:407] Looking for the following rc files: /etc/bazel.bazelrc,/home/dsun/work/gateway/plugins/wasm-cpp/.bazelrc,/home/dsun/.bazelrc [INFO 19:47:06.808 src/main/cpp/rc_file.cc:56] Parsing the RcFile /home/dsun/work/gateway/plugins/wasm-cpp/.bazelrc [INFO 19:47:06.808 src/main/cpp/rc_file.cc:56] Parsing the RcFile /home/dsun/.bazelrc [INFO 19:47:06.808 src/main/cpp/blaze.cc:1628] Debug logging requested, sending all client log statements to stderr [INFO 19:47:06.808 src/main/cpp/blaze.cc:1511] Acquired the client lock, waited 0 milliseconds [INFO 19:47:06.811 src/main/cpp/blaze.cc:1704] Trying to connect to server (timeout: 30 secs)... [INFO 19:47:06.857 src/main/cpp/blaze.cc:1266] Connected (server pid=1573542). [INFO 19:47:06.857 src/main/cpp/blaze.cc:1980] Releasing client lock, let the server manage concurrent requests. WARNING: Option 'host_javabase' is deprecated WARNING: Option 'javabase' is deprecated
Traceback (most recent call last): File "/home/dsun/.cache/bazel/_bazel_dsun/c4e768d90cc77b1af937ef4d90fbb834/execroot/istio_ecosystem_wasm_extensions/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/istio_ecosystem_wasm_extensions/refresh_compile_commands.check_python_version.py", line 15, in <module>
refresh_compile_commands.main()
File "/home/dsun/.cache/bazel/_bazel_dsun/c4e768d90cc77b1af937ef4d90fbb834/execroot/istio_ecosystem_wasm_extensions/bazel-out/k8-fastbuild/bin/refresh_compile_commands.py", line 1425, in main
compile_command_entries.extend(_get_commands(target, flags))
File "/home/dsun/.cache/bazel/_bazel_dsun/c4e768d90cc77b1af937ef4d90fbb834/execroot/istio_ecosystem_wasm_extensions/bazel-out/k8-fastbuild/bin/refresh_compile_commands.py", line 1286, in _get_commands
yield from _convert_compile_commands(parsed_aquery_output)
File "/home/dsun/.cache/bazel/_bazel_dsun/c4e768d90cc77b1af937ef4d90fbb834/execroot/istio_ecosystem_wasm_extensions/bazel-out/k8-fastbuild/bin/refresh_compile_commands.py", line 1166, in _convert_compile_commands
for source_files, header_files, compile_command_args in outputs:
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
yield _result_or_cancel(fs.pop())
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
return fut.result(timeout)
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/dsun/.cache/bazel/_bazel_dsun/c4e768d90cc77b1af937ef4d90fbb834/execroot/istio_ecosystem_wasm_extensions/bazel-out/k8-fastbuild/bin/refresh_compile_commands.py", line 1126, in _get_cpp_command_for_files
compile_action.arguments = _emscripten_platform_patch(compile_action)
File "/home/dsun/.cache/bazel/_bazel_dsun/c4e768d90cc77b1af937ef4d90fbb834/execroot/istio_ecosystem_wasm_extensions/bazel-out/k8-fastbuild/bin/refresh_compile_commands.py", line 828, in _emscripten_platform_patch
assert sysroot, f'Emscripten sysroot not detected in CMD: {compile_action.arguments}'
AssertionError: Emscripten sysroot not detected in CMD: ['external/proxy_wasm_cpp_sdk/toolchain/emcc.sh', '-MD', '-MF', 'bazel-out/wasm32-fastbuild-ST-4bf3c0394d2a/bin/extensions/sni_misdirect/_objs/_wasm_sni_misdirect.wasm/plugin.d', '-frandom-seed=bazel-out/wasm32-fastbuild-ST-4bf3c0394d2a/bin/extensions/sni_misdirect/_objs/_wasm_sni_misdirect.wasm/plugin.o', '-iquote', '.', '-iquote', 'bazel-out/wasm32-fastbuild-ST-4bf3c0394d2a/bin', '-iquote', 'external/com_google_absl', '-iquote', 'bazel-out/wasm32-fastbuild-ST-4bf3c0394d2a/bin/external/com_google_absl', '-iquote', 'external/proxy_wasm_cpp_sdk', '-iquote', 'bazel-out/wasm32-fastbuild-ST-4bf3c0394d2a/bin/external/proxy_wasm_cpp_sdk', '-iquote', 'external/bazel_tools', '-iquote', 'bazel-out/wasm32-fastbuild-ST-4bf3c0394d2a/bin/external/bazel_tools', '-std=c++17', '-no-canonical-prefixes', '-O3', '-ffunction-sections', '-fdata-sections', '-flto', '-c', 'extensions/sni_misdirect/plugin.cc', '-o', 'bazel-out/wasm32-fastbuild-ST-4bf3c0394d2a/bin/extensions/sni_misdirect/_objs/_wasm_sni_misdirect.wasm/plugin.o']
WASM_SDK
PROXY_WASM_CPP_HOST_SHA = "f38347360feaaf5b2a733f219c4d8c9660d626f0"
PROXY_WASM_CPP_HOST_SHA256 = "bf10de946eb5785813895c2bf16504afc0cd590b9655d9ee52fb1074d0825ea3"
http_archive(
name = "proxy_wasm_cpp_host",
sha256 = PROXY_WASM_CPP_HOST_SHA256,
strip_prefix = "proxy-wasm-cpp-host-" + PROXY_WASM_CPP_HOST_SHA,
url = "https://github.com/proxy-wasm/proxy-wasm-cpp-host/archive/" + PROXY_WASM_CPP_HOST_SHA +".tar.gz",
)
@cpsauer I can't find EM_BIN_PATH declared in emscripten_toolchain 2.0.7 which my proxy_wasm_cpp_sdk dependes on.
Hello @dantesun,
Thanks for providing the information. That helps me a lot to figure out the issue.
When I implemented the emscripten support, I expected your projects to use emsdk's official Bazel toolchain, so this tool is relying on sysroot flag that is supposed to be set inside that toolchain in order to make up the environment variable that is necessary to invoke emcc.sh wrapper script. However, it seems that your version of proxy_wasm_cpp_sdk implements its own Bazel toolchain, and it doesn't set sysroot flag.
I think we have two options to resolve this issue:
Since recent versions of proxy_wasm_cpp_sdk moved out of its own toolchain to emsdk's one, you can try to update your proxy_wasm_cpp_sdk and see if it resolves the issue.
If upgrade is not an option, we could then make a simple patch to this tool so that if sysroot flag is not found, it will not set EM_BIN_PATH environment variable. This is because proxy_wasm_cpp_sdk's emcc.sh wrapper doesn't seem to require any environment variable to be set.
(Thanks so much, Kon!)
I am working on istio wasm-cpp extensions project. And I am using the following bazel configuration:
When I run
bazel run @hedron_compile_commands//:refresh_all
, I got the error:I found that I have to specify the 'sysroot' to Bazel C++ compiler options, so I downloaed the Clang SDK from
https://github.com/llvm/llvm-project/releases/download/llvmorg-17.0.5/clang+llvm-17.0.5-x86_64-linux-gnu-ubuntu-22.04.tar.xz
, unpacked into/home/dsun/apps/clang
. Then I specify the 'sysroot' inbazel.rc
Then I ran
bazel run @hedron_compile_commands//:refresh_all
again and I got the following error:I got stuck and have no idea how to solve this. Please help.