search

nickdiego / compiledb

Tool for generating Clang's JSON Compilation Database files for make-based build systems.
GNU General Public License v3.0
1.38k stars 123 forks source link

differenet database generated while using parallel build(-jN) #72

Open kaijajan opened 5 years ago

kaijajan commented 5 years ago

I noticed different compilation database generated while using parallel build. I use the following commands to generate build_log.txt

make -w | tee build_log.txt
compiledb < build_log.txt

make -w -j8 | tee build_log.txt
compiledb < build_log.txt

I fetch part of the built log did not generated in compile_commands.json and use both of them run with compiledb. compiledb generates nothing between squares if build log is parallel one. but done well with normal one.

part of built log without -jN, build_log_j1.txt

make[2]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli'
cp -f -R man /home/user/workspaces/bsp/projects/ats2/target/mnt/
make[3]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
armv5-linux-gcc -c -I/home/user/workspaces/bsp/projects/ats2/app/include  -Os -Wall cli.c
armv5-linux-gcc -c -I/home/user/workspaces/bsp/projects/ats2/app/include  -Os -Wall cmd.c
armv5-linux-gcc -o cli cli.o cmd.o -Os -Wall    -L/home/user/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-strip cli
armv5-linux-gcc -c -I/home/user/workspaces/bsp/projects/ats2/app/include  -Os -Wall cliReadFile.c
armv5-linux-gcc -o cliReadFile cliReadFile.o cmd.o -Os -Wall    -L/home/user/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-strip cliReadFile
make[3]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli'
make[2]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli'
make[3]: Entering directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
for i in cli cliReadFile ; do \
    cp -f $i /home/user/workspaces/bsp/projects/ats2/target/bin || exit 1; \
done
make[3]: 'install' is up to date.
make[3]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/user/workspaces/bsp/projects/ats2/app/cli'

part of built log with -jN, build_log_j8.txt

make[2]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
cp -f -R man /home/archer/workspaces/bsp/projects/ats2/target/mnt/
make[3]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
armv5-linux-gcc -c -I/home/archer/workspaces/bsp/projects/ats2/app/include  -Os -Wall cli.c
armv5-linux-gcc -c -I/home/archer/workspaces/bsp/projects/ats2/app/include  -Os -Wall cmd.c
armv5-linux-gcc -c -I/home/archer/workspaces/bsp/projects/ats2/app/include  -Os -Wall cliReadFile.c
armv5-linux-gcc -o cli cli.o cmd.o -Os -Wall    -L/home/archer/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-gcc -o cliReadFile cliReadFile.o cmd.o -Os -Wall    -L/home/archer/workspaces/bsp/projects/ats2/target/usr/lib -lmisc -lperiphery -lcommon -lobj -lreadline
armv5-linux-strip cli
armv5-linux-strip cliReadFile
make[3]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
make[2]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
make[3]: Entering directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'
for i in cli cliReadFile ; do \
    cp -f $i /home/archer/workspaces/bsp/projects/ats2/target/bin || exit 1; \
done
make[3]: 'install' is up to date.
make[3]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli/src'
make[2]: Leaving directory '/home/archer/workspaces/bsp/projects/ats2/app/cli'

here are the commands i use to generate database

compiledb -p build_log_j1.txt -o j1.json
compiledb -p build_log_j8.txt -o j8.json

two files are basically the same except the order

RPCMoritz commented 5 years ago

I think this is due to parallel builds generating a disordered output. See how below all -c/-o/strip are clustered, whereas above they are in the correct order for each artifact. If you use -Bwnk flags for make, you should be able to generate the output with -j1 in a reasonable amount of time - therefore I would recommend that you avoid unnecessary concurrency. compiledb is quite fragile when it comes to parsing those make outputs.

lyonlh commented 4 years ago

@kaijajan The make's option --output-sync would help you.