search

rizsotto / Bear

Bear is a tool that generates a compilation database for clang tooling.
GNU General Public License v3.0
4.79k stars 314 forks source link

The latest version(3.0.16) cannot capture all compilation units #417

Closed tr4v3ler closed 2 years ago

tr4v3ler commented 2 years ago

Describe the bug For the same code, using the same compilation command, 2223 compilation units were captured using bear in version 2.3.11, but only 2069 compilation units were captured using 3.0.16, which obviously lost some compilation units.

To Reproduce Steps to reproduce the behavior: Use the source mtk_kernel, and the compilation command is as follows:

cd ~/mediatek_kernel
source ./env.sh
cd kernel-3.18
mkdir out
make O=out ARCH=arm64 tubads_defconfig
bear --library /usr/local/lib/bear/libexec.so -- make -j40 O=out ARCH=arm64

output from bear 2.3.11: compile_commands_2.3.11.zip output from bear 3.0.16: compile_commands_3.0.16.zip

Environment:

rizsotto commented 2 years ago

Hey @tr4v3ler, thanks for the ticket. Would you mind to attach the two outputs into this ticket, because reproduce it takes time (and I am working alone this project). Also the link you've passed requires authentication.

tr4v3ler commented 2 years ago

Hey @tr4v3ler, thanks for the ticket. Would you mind to attach the two outputs into this ticket, because reproduce it takes time (and I am working alone this project). Also the link you've passed requires authentication.

Sorry, I forgot to upload before. I'm on vacation now. I'll upload the previous output after the vacation.

TC500 commented 2 years ago

same problem, error msg :ERROR: ld.so: object '/usr/local/$LIB/bear/libexec.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. $LIB is empty, then the path '/usr/local//bear/libexec.so' is wrong, it should be '/usr/local/bear/libexec.so'

rizsotto commented 2 years ago

@TC500 your problem is not the same. (#416 is similar) Please read the INSTALL.md file.

TC500 commented 2 years ago

@TC500 your problem is not the same. (#416 is similar) Please read the INSTALL.md file.

@rizsotto follow INSTALL.md and not found something about '$LIB' the file /usr/local/bear/libexec.so exsist, but the path /usr/local/$LIB/bear/libexec.so is /usr/local//bear/libexec.so, not same with /usr/local/bear/libexec.so

rizsotto commented 2 years ago

Hey @TC500 , can we discuss your issue someplace else? (I would like to keep this issue about the linux kernel compilation database size difference between different Bear versions.) I recommend the #416 ticket to discuss it further, or open a thread in the discussion, or gitter has a a Bear channel too.

tr4v3ler commented 2 years ago

Hey @tr4v3ler, thanks for the ticket. Would you mind to attach the two outputs into this ticket, because reproduce it takes time (and I am working alone this project). Also the link you've passed requires authentication.

Hello, I'm back from vacation. I just uploaded the previous output.

annebell commented 2 years ago

I did a diff between these two outputs and found the following differences: The old version contains all the elements of the new version, plus these extra 154 entries.

[
  "arch/arm64/crypto/aes-ce-cipher.c",
  "arch/arm64/crypto/aes-ce.S",
  "arch/arm64/crypto/aes-glue.c",
  "arch/arm64/crypto/sha2-ce-core.S",
  "arch/arm64/crypto/sha2-ce-glue.c",
  "arch/arm64/kernel/asm-offsets.c",
  "arch/arm64/kernel/cputable.c",
  "arch/arm64/kernel/debug-monitors.c",
  "arch/arm64/kernel/entry-fpsimd.S",
  "arch/arm64/kernel/entry.S",
  "arch/arm64/kernel/fpsimd.c",
  "arch/arm64/kernel/head.S",
  "arch/arm64/kernel/hyp-stub.S",
  "arch/arm64/kernel/io.c",
  "arch/arm64/kernel/irq.c",
  "arch/arm64/kernel/process.c",
  "arch/arm64/kernel/ptrace.c",
  "arch/arm64/kernel/setup.c",
  "arch/arm64/kernel/signal.c",
  "arch/arm64/kernel/stacktrace.c",
  "arch/arm64/kernel/sys.c",
  "arch/arm64/kernel/time.c",
  "arch/arm64/kernel/vdso/gettimeofday.S",
  "arch/arm64/kernel/vdso/note.S",
  "arch/arm64/kernel/vdso/sigreturn.S",
  "arch/arm64/kernel/vdso/vdso.S",
  "arch/arm64/lib/bitops.S",
  "arch/arm64/lib/call_with_stack.S",
  "arch/arm64/lib/clear_page.S",
  "arch/arm64/lib/clear_user.S",
  "arch/arm64/lib/copy_from_user.S",
  "arch/arm64/lib/copy_in_user.S",
  "arch/arm64/lib/copy_page.S",
  "arch/arm64/lib/copy_to_user.S",
  "arch/arm64/lib/delay.c",
  "arch/arm64/lib/memchr.S",
  "arch/arm64/mm/cache.S",
  "arch/arm64/mm/context.c",
  "arch/arm64/mm/copypage.c",
  "arch/arm64/mm/dma-mapping.c",
  "arch/arm64/mm/extable.c",
  "arch/arm64/mm/fault.c",
  "arch/arm64/mm/flush.c",
  "arch/arm64/mm/init.c",
  "arch/arm64/mm/ioremap.c",
  "arch/arm64/mm/mmap.c",
  "arch/arm64/mm/mmu.c",
  "arch/arm64/mm/mt_cache_v8.S",
  "arch/arm64/mm/mt_innercache.c",
  "arch/arm64/mm/pageattr.c",
  "arch/arm64/mm/pgd.c",
  "arch/arm64/mm/proc.S",
  "block/bio.c",
  "block/blk-tag.c",
  "block/elevator.c",
  "crypto/api.c",
  "crypto/cipher.c",
  "crypto/compress.c",
  "crypto/crypto_wq.c",
  "crypto/memneq.c",
  "crypto/scatterwalk.c",
  "drivers/amba/bus.c",
  "fs/char_dev.c",
  "fs/file_table.c",
  "fs/open.c",
  "fs/read_write.c",
  "fs/stat.c",
  "fs/super.c",
  "init/calibrate.c",
  "init/do_mounts.c",
  "init/do_mounts_initrd.c",
  "init/init_task.c",
  "init/initramfs.c",
  "init/main.c",
  "init/noinitramfs.c",
  "kernel/bounds.c",
  "kernel/capability.c",
  "kernel/cpu.c",
  "kernel/exec_domain.c",
  "kernel/exit.c",
  "kernel/fork.c",
  "kernel/kmod.c",
  "kernel/panic.c",
  "kernel/ptrace.c",
  "kernel/resource.c",
  "kernel/softirq.c",
  "kernel/sysctl.c",
  "kernel/sysctl_binary.c",
  "kernel/user.c",
  "mm/filemap.c",
  "net/802/p8022.c",
  "net/802/psnap.c",
  "out/scripts/dtc/dtc-lexer.lex.c",
  "out/scripts/dtc/dtc-parser.tab.c",
  "scripts/dtc/checks.c",
  "scripts/dtc/data.c",
  "scripts/dtc/dtc.c",
  "scripts/dtc/flattree.c",
  "scripts/dtc/fstree.c",
  "scripts/dtc/livetree.c",
  "scripts/dtc/srcpos.c",
  "scripts/dtc/treesource.c",
  "scripts/dtc/util.c",
  "scripts/kallsyms.c",
  "scripts/mod/devicetable-offsets.c",
  "scripts/mod/empty.c",
  "scripts/mod/file2alias.c",
  "scripts/mod/mk_elfconfig.c",
  "scripts/mod/modpost.c",
  "scripts/mod/sumversion.c",
  "scripts/pnmtologo.c",
  "scripts/selinux/genheaders/genheaders.c",
  "scripts/selinux/mdp/mdp.c",
  "scripts/sortextable.c",
  "security/capability.c",
  "security/commoncap.c",
  "security/integrity/iint.c",
  "security/integrity/integrity_audit.c",
  "security/keys/gc.c",
  "security/keys/key.c",
  "security/keys/keyctl.c",
  "security/keys/keyring.c",
  "security/keys/permission.c",
  "security/keys/proc.c",
  "security/keys/process_keys.c",
  "security/keys/request_key.c",
  "security/keys/request_key_auth.c",
  "security/keys/sysctl.c",
  "security/keys/user_defined.c",
  "security/lsm_audit.c",
  "security/min_addr.c",
  "security/security.c",
  "security/selinux/avc.c",
  "security/selinux/exports.c",
  "security/selinux/netif.c",
  "security/selinux/netlink.c",
  "security/selinux/netnode.c",
  "security/selinux/netport.c",
  "security/selinux/nlmsgtab.c",
  "security/selinux/selinuxfs.c",
  "security/selinux/ss/avtab.c",
  "security/selinux/ss/conditional.c",
  "security/selinux/ss/ebitmap.c",
  "security/selinux/ss/hashtab.c",
  "security/selinux/ss/mls.c",
  "security/selinux/ss/policydb.c",
  "security/selinux/ss/services.c",
  "security/selinux/ss/sidtab.c",
  "security/selinux/ss/status.c",
  "security/selinux/ss/symtab.c",
  "sound/sound_core.c",
  "sound/sound_firmware.c",
  "usr/gen_init_cpio.c",
  "usr/initramfs_data.S"
]
rizsotto commented 2 years ago

Thanks @annebell for doing the diff!!!

@tr4v3ler I've noticed that the compilations happens in two different directories. (One in mtk_kernel/kernel-3.18 the other is in mtk_kernel_test/kernel-3.18.) Is it possible that these two kernel sources are different? Or the build is configured to build different modules?

tr4v3ler commented 2 years ago

Thanks @annebell for doing the diff!!!

@tr4v3ler I've noticed that the compilations happens in two different directories. (One in mtk_kernel/kernel-3.18 the other is in mtk_kernel_test/kernel-3.18.) Is it possible that these two kernel sources are different? Or the build is configured to build different modules?

The code is the same, mtk_kernel_test is the name of the target directory when copying.

tr4v3ler commented 2 years ago

Thanks @annebell for doing the diff!!!

@tr4v3ler I've noticed that the compilations happens in two different directories. (One in mtk_kernel/kernel-3.18 the other is in mtk_kernel_test/kernel-3.18.) Is it possible that these two kernel sources are different? Or the build is configured to build different modules?

I downloaded the code again and completed two clean compilations. This time, the result was the same! It's weird. Maybe I forgot to delete the out directory before compiling. I'm so sorry for wasting everyone's time. Maybe it's not a bug.