I see that bear use "grpc", does it means bear works in asynchronous mode？

[zhangxiaobin2]

I know that in the old version, bear capture the build command information during user build parse, record them to temp files, filter them and write to json file at the end of user build.

Today, I find "gprc" package that bear build dependencies. Does it means that bear works in asynchronous mode now? Does it capture build command information and send that information to the server to deal immediately？

If so, it can avoids deleting directories and files during user compilation.

Thanks...

[rizsotto]

Hi @zhangxiaobin2 , in 3.0 release Bear uses process wrappers, which are report process start, signal and stop events via gRPC. The process which collects these events (currently keeps all events in memory and) does write a single file at the end of the build. This file is the input for filtering the compiler calls and formating the JSON compilation database.

At process level, Bear is decomposed into 3 processes. intercept is a program, which intercepts all executions. It uses the LD_PRELOAD functionality of the dynamic linker, or when that is not supported it uses compiler wrappers. citnames is a programm, which takes an execution report and generates the JSON compilation database. bear is calling these processes one after the other.

The final JSON compilation database is still updated only once at the end of the build. So, if you mean asynchronous on the output file, it's still synchronous... This can be changed later releases if desired.

I don't get the "avoid deleting directories and files during user compilation" part of your message. Can you clarify what did mean by that?

[zhangxiaobin2]

Thanks for your reply。

I could understand the way you change to GRPC. Bear change the way for record compilation information from write temp files under /tmp/intercept-xxxxxxx to send event GRPC server, other data processing procedures are basically the same.

Let me explain "avoid deleting directories and files during user compilation". Some times, user would copy some files to temp directory for build, and delete them after build. Something like:

cp -r  test.c  build_temp/test.c
gcc -c build_temp/test.c
rm -rf  build_temp

So, we can't process the file "build_temp/test.c" after we get the compilation json file for clang。 I know this is a difficult problem，even unsolved:)

By the way, let me ask one more question. Can we hook the GRPC process to get some other information (like compiler pre-defined Macro) by ourself? :)

[rizsotto]

Thanks for clarifying the file deletion problem...

Bear filters out those entries where the file is not accessible. (It's doing it, so tools can trust the JSON compilation database has only valid entries.)

What I've seen elsewhere (Clang's scan-build is doing this) that it is possible to put custom logic into the compiler wrappers. (So, it calls the compiler and then the static analyzer in their case.) The problem with that is that you have a report which contains reference to a module which not exists in the file system. It also hard to make it as a generic solution. (You need to find a way how to execute the custom logic. Then you need to collect the output of this custom logic and merge them into a single output.)

IMO the solution for this is to fix the build system! ;)

[rizsotto]

On the other question... I don't fully get it. :)

The gRPC is is used for two main reasons: supervise and intercept. The supervise part is helping process/compiler wrappers (of Bear) to set up the command line and the environment. The intercept part is collecting the execution events (started, stopped, signalled).

You want to get/query or override/redefine compiler pre-defined macros? To query you can do it independently after the build finished. (From the compilation database.) To override you can do by creating a simple compiler wrapper. (OpenMP compiler wrappers are adding extra compiler flags. CUDA also has compiler wrapper, but that has more complex logic.)

If you just want to see what commands were executed. You can grab the intercept command output. (It's a JSON file with all context of the command execution.)

Glad to help if you give more context what you are trying to achieve.

[rizsotto]

I hope I answered your questions. I'm closing this ticket, but feel free to ask more if you still have questions.

[zhangxiaobin2]

I‘m sorry for the late reply. Thanks for your help:)