search

google / ml-compiler-opt

Infrastructure for Machine Learning Guided Optimization (MLGO) in LLVM.
Apache License 2.0
636 stars 93 forks source link

error: Could not setup Inlining Advisor for the requested mode and/or options #18

Closed prasitaGit closed 2 years ago

prasitaGit commented 2 years ago

I'm trying to run the demo following this link: https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md Everything goes fine till I run the fx build (i run the commands in order provided in the link. So I run fx set core.x64 followed by the given arguments before that). fx set runs successfully. However, fx build gives me this error:

[2754/67050] CXX host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_shared.cc.o FAILED: host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_shared.cc.o ../../../llvm-install/bin/clang++ -MD -MF host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_shared.cc.o.d -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -I../.. -Ihost_x64/gen -I../../sdk -Ihost_x64/gen/sdk -I../../sdk/lib/fit-promise/include -I../../sdk/lib/fit/include -I../../sdk/lib/stdcompat/include -I../../zircon/system/public -fclang-abi-compat=13.0 -fcolor-diagnostics -fcrash-diagnostics-dir=clang-crashreports -Xclang -fembed-bitcode=all -ffp-contract=off --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -ffile-compilation-dir=. -no-canonical-prefixes -fomit-frame-pointer -fdata-sections -ffunction-sections -Os -mllvm -enable-ml-inliner=release -gdwarf-5 -Xclang -debug-info-kind=constructor -g3 -Wall -Wextra -Wconversion -Wextra-semi -Wimplicit-fallthrough -Wnewline-eof -Wstrict-prototypes -Wwrite-strings -Wno-sign-conversion -Wno-unused-parameter -Wnonportable-system-include-path -fvisibility=hidden -Werror -Wno-error=deprecated-declarations -Wa,--fatal-warnings --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -fPIE -fvisibility-inlines-hidden -stdlib=libc++ -std=c++17 -fno-exceptions -fno-rtti -stdlib=libc++ -c ../../sdk/lib/syslog/cpp/logging_backend_shared.cc -o host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_shared.cc.o error: Could not setup Inlining Advisor for the requested mode and/or options 1 error generated.

[2755/67050] CXX host_x64/obj/sdk/lib/syslog/cpp/cpp.log_settings.cc.o FAILED: host_x64/obj/sdk/lib/syslog/cpp/cpp.log_settings.cc.o ../../../llvm-install/bin/clang++ -MD -MF host_x64/obj/sdk/lib/syslog/cpp/cpp.log_settings.cc.o.d -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -I../.. -Ihost_x64/gen -I../../sdk -Ihost_x64/gen/sdk -I../../sdk/lib/fit-promise/include -I../../sdk/lib/fit/include -I../../sdk/lib/stdcompat/include -I../../zircon/system/public -fclang-abi-compat=13.0 -fcolor-diagnostics -fcrash-diagnostics-dir=clang-crashreports -Xclang -fembed-bitcode=all -ffp-contract=off --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -ffile-compilation-dir=. -no-canonical-prefixes -fomit-frame-pointer -fdata-sections -ffunction-sections -Os -mllvm -enable-ml-inliner=release -gdwarf-5 -Xclang -debug-info-kind=constructor -g3 -Wall -Wextra -Wconversion -Wextra-semi -Wimplicit-fallthrough -Wnewline-eof -Wstrict-prototypes -Wwrite-strings -Wno-sign-conversion -Wno-unused-parameter -Wnonportable-system-include-path -fvisibility=hidden -Werror -Wno-error=deprecated-declarations -Wa,--fatal-warnings --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -fPIE -fvisibility-inlines-hidden -stdlib=libc++ -std=c++17 -fno-exceptions -fno-rtti -stdlib=libc++ -c ../../sdk/lib/syslog/cpp/log_settings.cc -o host_x64/obj/sdk/lib/syslog/cpp/cpp.log_settings.cc.o error: Could not setup Inlining Advisor for the requested mode and/or options 1 error generated. [2757/67050] CXX host_x64/obj/sdk/lib/syslog/cpp/cpp.macros.cc.o FAILED: host_x64/obj/sdk/lib/syslog/cpp/cpp.macros.cc.o ../../../llvm-install/bin/clang++ -MD -MF host_x64/obj/sdk/lib/syslog/cpp/cpp.macros.cc.o.d -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -I../.. -Ihost_x64/gen -I../../sdk -Ihost_x64/gen/sdk -I../../sdk/lib/fit-promise/include -I../../sdk/lib/fit/include -I../../sdk/lib/stdcompat/include -I../../zircon/system/public -fclang-abi-compat=13.0 -fcolor-diagnostics -fcrash-diagnostics-dir=clang-crashreports -Xclang -fembed-bitcode=all -ffp-contract=off --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -ffile-compilation-dir=. -no-canonical-prefixes -fomit-frame-pointer -fdata-sections -ffunction-sections -Os -mllvm -enable-ml-inliner=release -gdwarf-5 -Xclang -debug-info-kind=constructor -g3 -Wall -Wextra -Wconversion -Wextra-semi -Wimplicit-fallthrough -Wnewline-eof -Wstrict-prototypes -Wwrite-strings -Wno-sign-conversion -Wno-unused-parameter -Wnonportable-system-include-path -fvisibility=hidden -Werror -Wno-error=deprecated-declarations -Wa,--fatal-warnings --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -fPIE -fvisibility-inlines-hidden -stdlib=libc++ -std=c++17 -fno-exceptions -fno-rtti -stdlib=libc++ -c ../../sdk/lib/syslog/cpp/macros.cc -o host_x64/obj/sdk/lib/syslog/cpp/cpp.macros.cc.o error: Could not setup Inlining Advisor for the requested mode and/or options 1 error generated.

[2758/67050] CXX host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_host.cc.o FAILED: host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_host.cc.o ../../../llvm-install/bin/clang++ -MD -MF host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_host.cc.o.d -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -I../.. -Ihost_x64/gen -I../../sdk -Ihost_x64/gen/sdk -I../../sdk/lib/fit-promise/include -I../../sdk/lib/fit/include -I../../sdk/lib/stdcompat/include -I../../zircon/system/public -fclang-abi-compat=13.0 -fcolor-diagnostics -fcrash-diagnostics-dir=clang-crashreports -Xclang -fembed-bitcode=all -ffp-contract=off --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -ffile-compilation-dir=. -no-canonical-prefixes -fomit-frame-pointer -fdata-sections -ffunction-sections -Os -mllvm -enable-ml-inliner=release -gdwarf-5 -Xclang -debug-info-kind=constructor -g3 -Wall -Wextra -Wconversion -Wextra-semi -Wimplicit-fallthrough -Wnewline-eof -Wstrict-prototypes -Wwrite-strings -Wno-sign-conversion -Wno-unused-parameter -Wnonportable-system-include-path -fvisibility=hidden -Werror -Wno-error=deprecated-declarations -Wa,--fatal-warnings --sysroot=../../prebuilt/third_party/sysroot/linux --target=x86_64-unknown-linux-gnu -fPIE -fvisibility-inlines-hidden -stdlib=libc++ -std=c++17 -fno-exceptions -fno-rtti -stdlib=libc++ -c ../../sdk/lib/syslog/cpp/logging_backend_host.cc -o host_x64/obj/sdk/lib/syslog/cpp/backend_host.logging_backend_host.cc.o error: Could not setup Inlining Advisor for the requested mode and/or options 1 error generated.

[2761/67050] CC efi_x64/obj/third_party/lz4/lib/liblz4.lz4hc.c.o ninja: build stopped: subcommand failed. Hint: run fx build with the option --log LOGFILE to generate a debug log if you are reporting a bug.

It'd be great to have some clarity on this.

mtrofin commented 2 years ago

That means that you're running a clang that doesn't have an inliner policy embedded. hmm... could you check you did this step, and that LLVM_INSTALLDIR_RELEASE is correct, and that in the fx set command, the clang_prefix points to the same place LLVM_INSTALLDIR_RELEASE points?

prasitaGit commented 2 years ago

I didn't go till that step. I'm stuck precisely here: https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md#build-fuchsia, i.e., before extracting the corpus.

mtrofin commented 2 years ago

Aaah... I think I see why. By default, Fuchsia builds now with the ml inliner enabled, but since the default clang doesn't have that built in, you need to disable it at this stage. Add to fx set --args=clang_ml_inliner=false to disable it.

When we wrote the demo, they haven't yet flipped the default. Please let me know if this unblocks you, and I'll update the demo.

Thanks!

prasitaGit commented 2 years ago

Thanks for the reply. I did what you said. The script looks like: cd ${FUCHSIA_SRCDIR} jiri update ~/ml-compiler-opt/docs/demo/fuchsia.xml fx set core.x64 \ --args='clang_prefix="/usr/local/google/home/mtrofin/llvm-install/bin"' \ --args=clang_embed_bitcode=true \ --args=clang_ml_inliner=false \ --args='optimize="size"' fx build

But now, fx set fails giving this error: [17:07:30.823] Jiri packages are not fetched due to fatal errors when updating projects. [17:07:30.823] Jiri hooks are not run due to fatal errors when updating projects or packages ERROR: cannot move "/home/saikat/MLGO/fuchsia/third_party/pyyaml/src" to "/home/saikat/MLGO/fuchsia/third_party/pyyaml" as the destination already exists ERROR at //BUILD.gn:201:5: Script returned non-zero exit code. exec_script("//prebuilt/third_party/gn/${host_platform}/gn", gn_cmd) ^---------- Current dir: /home/saikat/MLGO/fuchsia/out/default/ Command: /usr/bin/env /home/saikat/MLGO/fuchsia/prebuilt/third_party/gn/linux-x64/gn gen -q --root=../../zircon --args=# THIS FILE IS CLOBBERED. DO NOT EDIT! Instead, edit //out/default/args.gn to add zircon_extra_args = { ... } to override settings below. asan_default_options = ["detect_stack_use_after_return=1"] lsan_default_options = [] ubsan_default_options = ["print_stacktrace=1", "halt_on_error=1"] scudo_default_options = [] forward_variables_from({ clang_tool_dir = "/home/saikat/MLGO/llvm-install/bin" default_deps = ["//:legacy_unification-x64"] disable_kernel_pci = false goma_dir = "/home/saikat/MLGO/fuchsia/prebuilt/third_party/goma/linux-x64" optimize = "size" output_gsym = false rustc_version_string = "XJj9WwQ5ZIufh8195356YT-rfrxfxUL44aixzXq6oL8C" use_ccache = false use_goma = false variants = [] zbi_compression = "zstd" zx_fidl_trace_level = 0 }, "*") --export-compile-commands=default ../default.zircon Returned 1 and printed out:

ERROR at //public/gn/BUILDCONFIG.gn:863:3: Dependency not allowed. group("${_library_name}.headers") { ^---------------------------------- The item //system/utest/abi-type-validator:abi-type-validator.headers can not depend on //system/utest/abi-type-validator:_library.config.abi-type-validator because it is not in //system/utest/abi-type-validator:_library.config.abi-type-validator's visibility list: [ //system/utest/abi-type-validator/abi-type-validator.headers:abi-type-validator.headers ]

See //BUILD.gn:203:3: whence it was called. _run_zircon_gn("zircon") {

mtrofin commented 2 years ago

when doing jiri update, pass -gc, like this: jiri update -gc ~/ml-compiler-opt/docs/demo/fuchsia.xml

also make sure to use the llvm hash specified there.

Sorry about this - we should update the demo to more recent fuchsia/llvm - they seem to have gotten out of sync if you update llvm but not fuchsia; I'll chat with the Fuchsia folks on Tuesday and get it up to date.

prasitaGit commented 2 years ago

Thanks for the reply. Where exactly should I use the llvm hash? Also on including -gc like you said, I get load error in fx set:

ERROR at //third_party/cobalt/src/lib/util/BUILD.gn:8:1: Can't load input file. import("//third_party/protobuf/proto_library.gni") ^------------------------------------------------ Unable to load: /home/saikat/MLGO/fuchsia/third_party/protobuf/proto_library.gni I also checked in the secondary tree for: /home/saikat/MLGO/fuchsia/build/secondary/third_party/protobuf/proto_library.gni See //src/cobalt/bin/app/BUILD.gn:79:5: which caused the file to be included. "//third_party/cobalt/src/lib/util:pem_util", ^------------------------------------------- ERROR: error running gn gen: exit status 1 Re-running gn gen first (/home/saikat/MLGO/fuchsia/prebuilt/third_party/gn/linux-x64/gn changed) ERROR at //third_party/cobalt/src/logger/BUILD.gn:5:1: Can't load input file. import("//third_party/protobuf/proto_library.gni") ^------------------------------------------------ Unable to load: /home/saikat/MLGO/fuchsia/third_party/protobuf/proto_library.gni I also checked in the secondary tree for: /home/saikat/MLGO/fuchsia/build/secondary/third_party/protobuf/proto_library.gni See //src/cobalt/bin/app/BUILD.gn:81:5: which caused the file to be included. "//third_party/cobalt/src/logger",

mtrofin commented 2 years ago

May be worth deleting the fuchsia source dir and then fetching it again - I'm assuming you followed this, and ended up doing a curl -s "https://fuchsia.googlesource.com/fuchsia/+/HEAD/scripts/bootstrap?format=TEXT" | base64 --decode | bash, correct?

so after that if you do cd ${FUCHSIA_SRCDIR} && jiri update -gc ~/ml-compiler-opt/docs/demo/fuchsia.xml, it should get you in a usable state.

The llvm hash is the git hash - i.e. after you clone llvm, cd llvm-project && git checkout fa4c3f70ff0768a270b0620dc6d158ed1205ec4e

Some background: Fuchsia is really a suite of repos, and the xml doc specifies a snapshot of all of them, which defines the product (similar to how chrome, for example, ingests a bunch of repos, like v8). Fuchsia also periodically fetches llvm from upstream and builds and tests their own compiler toolchain (hence the pretty custom cmake flags). As any of these repos evolve, they get out of sync (stuff breaks), hence our specifying the exact snapshot where we tried out the demo. That's also the reason we want to start updating the demo snapshot (i.e. the xml file and the llvm hash) more frequently, so folks can try things out with llvm close to HEAD (with high probability that if they also want to try it exactly at HEAD, it'd work).

(The only nuance that may cause some friction, though, right now, is that the IDK and SYSROOT packages aren't captured in the xml file, but that shouldn't manifest itself during jiri or fx set.)

prasitaGit commented 2 years ago

I did the above steps. I don't get the issues described above, most of the fx build runs but I get this: [17/3219] ACTION //sdk/lib/ui/scenic/cpp:gen_command_sizing(//build/toolchain/fuchsia:x64) FAILED: obj/sdk/lib/ui/scenic/cpp/scenic-measure-tape.checked /usr/bin/env ../../build/gn_run_binary.sh /homes/mukher39/scratch/llvm-install/bin host_x64/measure-tape --json fidling/gen/sdk/fidl/fuchsia.images/fuchsia.images.fidl.json --json fidling/gen/sdk/fidl/fuchsia.ui.gfx/fuchsia.ui.gfx.fidl.json --json fidling/gen/sdk/fidl/fuchsia.ui.input/fuchsia.ui.input.fidl.json --json fidling/gen/sdk/fidl/fuchsia.ui.scenic/fuchsia.ui.scenic.fidl.json --json fidling/gen/sdk/fidl/fuchsia.ui.views/fuchsia.ui.views.fidl.json --target-binding hlcpp --target-type fuchsia.ui.scenic/Command --out-h /u/scratch1/mukher39/fuchsia/sdk/lib/ui/scenic/cpp/commands_sizing.h --h-include-path lib/ui/scenic/cpp/commands_sizing.h --out-cc /u/scratch1/mukher39/fuchsia/sdk/lib/ui/scenic/cpp/commands_sizing.cc --only-check-to-file /u/scratch1/mukher39/fuchsia/out/default/obj/sdk/lib/ui/scenic/cpp/scenic-measure-tape.checked /u/scratch1/mukher39/fuchsia/sdk/lib/ui/scenic/cpp/commands_sizing.h and/or /u/scratch1/mukher39/fuchsia/sdk/lib/ui/scenic/cpp/commands_sizing.cc is out of date! Please run the following

./host_x64/measure-tape \ --json \ fidling/gen/sdk/fidl/fuchsia.images/fuchsia.images.fidl.json \ --json \ fidling/gen/sdk/fidl/fuchsia.ui.gfx/fuchsia.ui.gfx.fidl.json \ --json \ fidling/gen/sdk/fidl/fuchsia.ui.input/fuchsia.ui.input.fidl.json \ --json \ fidling/gen/sdk/fidl/fuchsia.ui.scenic/fuchsia.ui.scenic.fidl.json \ --json \ fidling/gen/sdk/fidl/fuchsia.ui.views/fuchsia.ui.views.fidl.json \ --target-binding \ hlcpp \ --target-type \ fuchsia.ui.scenic/Command \ --out-h \ /u/scratch1/mukher39/fuchsia/sdk/lib/ui/scenic/cpp/commands_sizing.h \ --h-include-path \ lib/ui/scenic/cpp/commands_sizing.h \ --out-cc \ /u/scratch1/mukher39/fuchsia/sdk/lib/ui/scenic/cpp/commands_sizing.cc

[48/3219] CXX x64-shared/obj/zircon/system/ulib/zxio/remote_v2/libzxio.remote_v2.cc.o ninja: build stopped: subcommand failed. I downloaded and installed fuchsia just and ran this. I can't comprehend how can commands_sizing.h be out of date? I'm not sure whether the issue is on Fuchsia's end or here.

mtrofin commented 2 years ago

I'm about to update the demo - it's very likely a bunch of instructions got old / packages out of sync (i.e. the IDK relative to fuchsia and llvm). The update would let you 'be' at HEAD of fuchsia and a fuchsia-determined git hash of llvm, close to llvm head. Trying it out locally first to make sure all holds up, I should have it updated by EOD today. I'll let you know. Sorry this is causing trouble.

mtrofin commented 2 years ago

Updated. The TL;DR; is that you can just jiri update -gc under ${FUCHSIA_SRCDIR} without needing to worry about that fuchsia.xml file. This will sync the fuchsia repos at HEAD. The benefit is that we now know we don't have to worry about the IDK / SYSROOT packages being out of sync.

From there, you jiri package 'fuchsia/third_party/clang${platform}' (verbatim - i.e. don't substitute ${platform}) and that gives you the right llvm repo hash to use (see https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md#set-up-the-correct-package-versions). Normally you could try work at llvm HEAD, too, but right now there were some libc++ updates in the llvm repo that fuchsia didn't yet pick up, so if you leave llvm at HEAD, you'll see later some errors when building fuchsia, about std::sort not being found. The hash that the jiri package command above gives you is what the fuchsia build bots use, too.

One thing that may have also caused you trouble: the build system (cmake/ninja) isn't quite hermetic, i.e. the best thing to do after updating the repos is to delete the build dirs (i.e. cd ${FUCHSIA_SRCDIR} && rm -rf out/default and cd ${LLVM_SRCDIR} && rm -rf build. Then re-do creating the build dir for llvm, cmake... all that (and when you get to it, same for fx set).

prasitaGit commented 2 years ago

Thanks for the update. I did what you said, and followed the steps in the updated demo. Both the hashes are in sync. When I run fx build, it fails in a later stage giving me this error:

[44623/55563] ACTION //garnet/bin/sched:sched.verify(//build/toolchain/fuchsia:x64) FAILED: gen/garnet/bin/sched/sched.verify ../../prebuilt/third_party/python3/linux-x64/bin/python3.8 -S ../../build/dist/verify_manifest_elf_binaries.py --check-stripped --depfile=gen/garnet/bin/sched/sched.verify.d --stamp=gen/garnet/bin/sched/sched.verify --fini-manifest=obj/garnet/bin/sched/sched_manifest --check-unstripped-files --toolchain-lib-dir=../../../../../../homes/mukher39/scratch/llvm-install/bin/../lib --toolchain-lib-dir=../../prebuilt/third_party/rust/linux-x64/bin/../lib ERRORS FOUND IN obj/garnet/bin/sched/sched_manifest: bin/sched missing dependency lib/libc++.so.2 bin/sched missing dependency lib/libc++abi.so.1 Binary bin/sched has interp ld.so.1, lib_dir lib/

I'm using python3.8.

mtrofin commented 2 years ago

The python version shouldn't matter... (and 3.8 is good). The libc++ part is odd though. Let's do a few things to try narrow it down.

could you, in fuchsia:

rm -rf out/default
fx set core.x64 \
  --args=clang_embed_bitcode=true \
  --args='optimize="size"' \
  --args='clang_ml_inliner=false'
fx build

Let's see first if fuchsia, built with its toolchain, goes well.

Then, could you paste the output from jiri package 'fuchsia/third_party/clang/${platform}' and then, cd ${LLVM_SRCDIR} and git status and paste the output there?

btw, this may be better / easier to do interactively, if you want, ping me on chat (google meet) - same alias I use here, at google.com.

prasitaGit commented 2 years ago

it worked! I think I was doing jiri update -gc before fx set, hence it was giving the error. I couldn't have managed to resolve this without your constant help. i'd do the rest of the procedure now. If I run into issues, I'll ping you on google meet if that's okay

mtrofin commented 2 years ago

Doing jiri update -gc before fx set makes sense though - but no need to do it again (i.e. it just updates the fuchsia repo, and now it's updated to head, so let's keep that as-is)

If the above worked, it means the llvm repo is out of sync - all we did above is tell the fuchsia build system not to use the built-in llvm, but that's not what (I assume) you want.

Can you paste those 2 bits of info - the jiri package output and the git status, let's see if anything obvious pops.

prasitaGit commented 2 years ago

jiri package output:

llvm:

HEAD detached at 1aa59ff2f789

mtrofin commented 2 years ago

ah... do you see a nothing to commit, working tree clean line? Mine looks like this:

$ git status
HEAD detached at 1aa59ff2f789
nothing to commit, working tree clean

$
prasitaGit commented 2 years ago

Yes, it does give that. This is the entire output:

HEAD detached at 1aa59ff2f789

It took 29.24 seconds to enumerate untracked files. 'status -uno' may speed it up, but you have to be careful not to forget to add new files yourself (see 'git help status'). nothing to commit, working tree clean

mtrofin commented 2 years ago

hmm... could you do a git status -u mine does the same as before:

$ git status -u
HEAD detached at 1aa59ff2f789
nothing to commit, working tree clean

$

Now, if that starts spewing a ton of stuff, do a git clean -df to delete the untracked files.

prasitaGit commented 2 years ago

it gives the same result as above

mtrofin commented 2 years ago

OK, so I understand where we are: did you re-do the demo:

prasitaGit commented 2 years ago

Yes this is resolved. Hence, I deleted the comment.

prasitaGit commented 2 years ago

But module_paths is empty in my case, hence it gives 0 of 0 modules succeeded. I'd like to point out that fx compdb doesn't work. It shows: DEPRECATED: fx now automatically creates and tracks one's compilation database; this command will soon be deleted. I don't believe this would cause module_paths to be empty though

prasitaGit commented 2 years ago

nvm, I think i missed the --clang-prefix while doinf fx set. I'm redoing the build again. I was trying exactly this:

rm -rf out/default fx set core.x64 \ --args=clang_embed_bitcode=true \ --args='optimize="size"' \ --args='clang_ml_inliner=false' fx build

mtrofin commented 2 years ago

OK, that is a separate problem. for the modules_path, replace the -Oz with -Os in the call to extract_ir.py (I'll fix this asap in the doc, forgot it it seems)

(Background: that's just a filter, and Fuchsia recently moved to use -Os rather than -Oz.)

But that aside, yes, we need to: wipe out the build dir under $LLVM_SRCDIR, and re-do https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md#build-llvm, then re-build fuchsia (delete out/default to be sure)

prasitaGit commented 2 years ago

I did the steps, i.e., removed the build folder, built it again, then removed out/default in fuchsia and ran fx set with --clang-prefix (which I didn't do previously, when it ran successfully). It gives me this:

FAILED: gen/garnet/bin/sched/sched.verify ../../prebuilt/third_party/python3/linux-x64/bin/python3.8 -S ../../build/dist/verify_manifest_elf_binaries.py --check-stripped --depfile=gen/garnet/bin/sched/sched.verify.d --stamp=gen/garnet/bin/sched/sched.verify --fini-manifest=obj/garnet/bin/sched/sched_manifest --check-unstripped-files --toolchain-lib-dir=../../../../../../homes/mukher39/scratch/llvm-install/bin/../lib --toolchain-lib-dir=../../prebuilt/third_party/rust/linux-x64/bin/../lib ERRORS FOUND IN obj/garnet/bin/sched/sched_manifest: bin/sched missing dependency lib/libc++.so.2 bin/sched missing dependency lib/libc++abi.so.1 Binary bin/sched has interp ld.so.1, lib_dir lib/ [23572/34490] RUST bt_gap exe.unstripped/bt_gap

I installed lld from the present git version in llvm_project and tried the same steps again. Now I get this in fx set: [23541/34490] ACTION //garnet/bin/ktrace_provider:ktrace_provider.verify(//build/toolchain/fuchsia:x64) FAILED: gen/garnet/bin/ktrace_provider/ktrace_provider.verify ../../prebuilt/third_party/python3/linux-x64/bin/python3.8 -S ../../build/dist/verify_manifest_elf_binaries.py --check-stripped --depfile=gen/garnet/bin/ktrace_provider/ktrace_provider.verify.d --stamp=gen/garnet/bin/ktrace_provider/ktrace_provider.verify --fini-manifest=obj/garnet/bin/ktrace_provider/ktrace_provider_manifest --check-unstripped-files --toolchain-lib-dir=../../../../../../homes/mukher39/scratch/llvm-install/bin/../lib --toolchain-lib-dir=../../prebuilt/third_party/rust/linux-x64/bin/../lib ERRORS FOUND IN obj/garnet/bin/ktrace_provider/ktrace_provider_manifest: bin/ktrace_provider missing dependency lib/libc++.so.2 bin/ktrace_provider missing dependency lib/libc++abi.so.1 Binary bin/ktrace_provider has interp ld.so.1, lib_dir lib/ [23547/34490] ACTION //garnet/bin/sched:sched.verify(//build/toolchain/fuchsia:x64) FAILED: gen/garnet/bin/sched/sched.verify ../../prebuilt/third_party/python3/linux-x64/bin/python3.8 -S ../../build/dist/verify_manifest_elf_binaries.py --check-stripped --depfile=gen/garnet/bin/sched/sched.verify.d --stamp=gen/garnet/bin/sched/sched.verify --fini-manifest=obj/garnet/bin/sched/sched_manifest --check-unstripped-files --toolchain-lib-dir=../../../../../../homes/mukher39/scratch/llvm-install/bin/../lib --toolchain-lib-dir=../../prebuilt/third_party/rust/linux-x64/bin/../lib ERRORS FOUND IN obj/garnet/bin/sched/sched_manifest: bin/sched missing dependency lib/libc++.so.2 bin/sched missing dependency lib/libc++abi.so.1 Binary bin/sched has interp ld.so.1, lib_dir lib/ [23572/34490] RUST bt_gap exe.unstripped/bt_gap ninja: build stopped: subcommand failed.v

mtrofin commented 2 years ago

Hmm... try deleting your $LLVM_INSTALLDIR and re-doing the install step - i.e. (from https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md#build-llvm):

DESTDIR=${LLVM_INSTALLDIR} ninja install-distribution-stripped
cd ${FUCHSIA_SRCDIR}
python scripts/clang/generate_runtimes.py --clang-prefix=$LLVM_INSTALLDIR --sdk-dir=$IDK_DIR --build-id-dir=$LLVM_INSTALLDIR/lib/.build-id > $LLVM_INSTALLDIR/lib/runtime.json
prasitaGit commented 2 years ago

It gives the same error:

[44474/55563] ACTION //garnet/bin/sched:sched.verify(//build/toolchain/fuchsia:x64) FAILED: gen/garnet/bin/sched/sched.verify ../../prebuilt/third_party/python3/linux-x64/bin/python3.8 -S ../../build/dist/verify_manifest_elf_binaries.py --check-stripped --depfile=gen/garnet/bin/sched/sched.verify.d --stamp=gen/garnet/bin/sched/sched.verify --fini-manifest=obj/garnet/bin/sched/sched_manifest --check-unstripped-files --toolchain-lib-dir=../../../../../../homes/mukher39/scratch/llvm-install/bin/../lib --toolchain-lib-dir=../../prebuilt/third_party/rust/linux-x64/bin/../lib ERRORS FOUND IN obj/garnet/bin/sched/sched_manifest: bin/sched missing dependency lib/libc++.so.2 bin/sched missing dependency lib/libc++abi.so.1 Binary bin/sched has interp ld.so.1, lib_dir lib/ [44505/55563] RUST bt_gap exe.unstripped/bt_gap ninja: build stopped: subcommand failed.

mtrofin commented 2 years ago

Huh. OK, can you do (from ${FUCHSIA_SRCDIR}) a jiri snapshot /tmp/snapshot.xml, then send me that xml file - maybe as a gist or something - I'll try to repro your setup locally.

prasitaGit commented 2 years ago

It is a very big file. I don't know what an apt gist maybe. But this is the entire file snapshot.xml.zip

mtrofin commented 2 years ago

Thanks - just off chance, are you doing this on a linux box, or a mac box?

prasitaGit commented 2 years ago

linux.

mtrofin commented 2 years ago

Huh... ok, so that's not the problem then (i.e. we're also on linux - a debian distro, fwiw, but I doubt that's it)

Well, "it works on my machine"... let's look at something, what's your out/default/obj/garnet/bin/sched/sched_manifest look like?

Mine:

bin/sched=sched
lib/ld.so.1=user.libc_x64/libc.so
lib/libasync-default.so=x64-shared/libasync-default.so
lib/libbackend_fuchsia_globals.so=x64-shared/libbackend_fuchsia_globals.so
lib/libc++.so.2=../../../llvm-install/lib/.build-id/a6/330277eae2394d
lib/libc++abi.so.1=../../../llvm-install/lib/.build-id/fe/357bd2932b67f1
lib/libfdio.so=x64-shared/libfdio.so
lib/libsyslog.so=x64-shared/libsyslog.so
lib/libunwind.so.1=../../../llvm-install/lib/.build-id/a2/be6e87187bf4d1
meta/package=gen/garnet/bin/sched/sched_meta_package.txt
prasitaGit commented 2 years ago

It gives me this:

bin/sched=sched lib/ld.so.1=user.libc_x64/libc.so lib/libasync-default.so=x64-shared/libasync-default.so lib/libbackend_fuchsia_globals.so=x64-shared/libbackend_fuchsia_globals.so lib/libfdio.so=x64-shared/libfdio.so lib/libsyslog.so=x64-shared/libsyslog.so meta/package=gen/garnet/bin/sched/sched_meta_package.txt

I see i don't have entries for libc++ which is present in your output.

mtrofin commented 2 years ago

Ah! we're getting somewhere (hopefully) :)

OK, umm... you just git clone llvm-project, right? like so:

git clone git@github.com:llvm/llvm-project.git

prasitaGit commented 2 years ago

Should I delete my present llvm-project repo then?

mtrofin commented 2 years ago

well... so we understand first where the difference comes from, did you clone it differently?

prasitaGit commented 2 years ago

I just followed the link posted in the demo. Precisely this: git clone https://github.com/llvm/llvm-project.git. Then I checked out with the hash I get from jiri.

mtrofin commented 2 years ago

ok, so that should be fine. So there must be something about the way this file is set up maybe, let me get our Fuchsia colleagues to take a look

mtrofin commented 2 years ago

and when you did this step: https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md#build-llvm

did you make sure to run the last line after the ninja install-distribution-stripped step?

prasitaGit commented 2 years ago

You mean this step right: python scripts/clang/generate_runtimes.py --clang-prefix=$LLVM_INSTALLDIR --sdk-dir=$IDK_DIR --build-id-dir=$LLVM_INSTALLDIR/lib/.build-id > $LLVM_INSTALLDIR/lib/runtime.json ? Yes, I did this.

mtrofin commented 2 years ago

ok... huh. just in case, can you delete $LLVM_INSTALLDIR/lib/runtime.json and redo (just in case readonly flags and quiet error messages)

prasitaGit commented 2 years ago

redo from the python step? Right after inja install-distribution-stripped?

mtrofin commented 2 years ago

right. maybe save the old one, redo it, then diff the 2?

mtrofin commented 2 years ago

by "the old one" I mean the runtime.json file - i.e. if they are not different, no point in rebuilding fuchsia after that :)

prasitaGit commented 2 years ago

They are not different. I checked using the diff command. This is the op:

[ { "cflags": [], "ldflags": [], "runtime": [], "target": [ "x86_64-unknown-fuchsia" ] }, { "cflags": [], "ldflags": [ "-static-libstdc++" ], "runtime": [], "target": [ "x86_64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=address" ], "ldflags": [], "runtime": [], "target": [ "x86_64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=address" ], "ldflags": [ "-static-libstdc++" ], "runtime": [], "target": [ "x86_64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=undefined" ], "ldflags": [], "runtime": [], "target": [ "x86_64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=undefined" ], "ldflags": [ "-static-libstdc++" ], "runtime": [], "target": [ "x86_64-unknown-fuchsia" ] }, { "cflags": [], "ldflags": [], "runtime": [], "target": [ "aarch64-unknown-fuchsia" ] }, { "cflags": [], "ldflags": [ "-static-libstdc++" ], "runtime": [], "target": [ "aarch64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=address" ], "ldflags": [], "runtime": [], "target": [ "aarch64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=address" ], "ldflags": [ "-static-libstdc++" ], "runtime": [], "target": [ "aarch64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=undefined" ], "ldflags": [], "runtime": [], "target": [ "aarch64-unknown-fuchsia" ] }, { "cflags": [ "-fsanitize=undefined" ], "ldflags": [ "-static-libstdc++" ], "runtime": [], "target": [ "aarch64-unknown-fuchsia" ] } ]

mtrofin commented 2 years ago

Oh! ok. Maybe that's the problem - it shouldn't be empty values. Mine looks like:


  {
    "cflags": [],
    "ldflags": [],
    "runtime": [
      {
        "debug": ".build-id/a6/330277eae2394d.debug",
        "dist": ".build-id/a6/330277eae2394d",
        "name": "libc++",
        "soname": "libc++.so.2"
      },
      {
        "debug": ".build-id/a2/be6e87187bf4d1.debug",
        "dist": ".build-id/a2/be6e87187bf4d1",
        "name": "libunwind",
        "soname": "libunwind.so.1"
      },
      {
        "debug": ".build-id/fe/357bd2932b67f1.debug",
        "dist": ".build-id/fe/357bd2932b67f1",
        "name": "libc++abi",
        "soname": "libc++abi.so.1"
      }
    ],
    "target": [
      "x86_64-unknown-fuchsia"
    ]
  },
  {
....

Couple of things coming to mind, when you run the python script that should generate this, are you using python3 (I remember you said you do, but sometimes python points to 2.7 and python3 is v3), also, are all the env vars the demo talks about defined?

prasitaGit commented 2 years ago

it is python3. I just checked python --version and it gave me Python 3.8.5. The variables are set. I believe these are the ones: $LLVM_SRCDIR is set to /homes/mukher39/scratch/llvm-project $LLVM_INSTALLDIR is set to /homes/mukher39/scratch/llvm-install $IDK_DIR is set to /homes/mukher39/scratch/fuchsia-idk $SYSROOT_DIR is set to /homes/mukher39/scratch/fuchsia-sysroot $FUCHSIA_SRCDIR is set to /homes/mukher39/scratch/fuchsia

These are all absolute paths. So there shouldn't be a problem in locating them with cd or cp.

mtrofin commented 2 years ago

Would you mind debugging the python script - I'm thinking this would be the fastest thing, as this seems to be somehow (and we can't figure out how) tied to your environment.

prasitaGit commented 2 years ago

Sure. I'd do that

mtrofin commented 2 years ago

Thanks - btw, if you find a bug, you can (if you want) patch it to Fuchsia; anyway, at minimum, you get the credit in the patch for finding it, of course!