Open whentojump opened 1 month ago
A post of relevant topic from Rust blog: https://blog.rust-lang.org/inside-rust/2020/11/12/source-based-code-coverage.html
Note: It claims gcov relies on debuginfo to correlate IR to source code. I think this is incorrect:
CONFIG_DEBUG_INFO
on eithersyzkaller document on their "covearge": https://github.com/google/syzkaller/blob/master/docs/coverage.md which is also the opposite of "source-based".
https://elixir.bootlin.com/linux/v6.10-rc7/source/net/netlink/genetlink.c#L264
GCC gcov report:
15898: 264: if ((flags & GENL_CMD_CAP_DO && !full->doit) ||
condition outcomes covered 8/8
7949: 265: (flags & GENL_CMD_CAP_DUMP && !full->dumpit)) {
Clang Source-based Code Coverage report:
264| 15.8k| if ((flags & GENL_CMD_CAP_DO && !full->doit) ||
265| 15.8k| (flags & GENL_CMD_CAP_DUMP && !full->dumpit)) {
------------------
|---> MC/DC Decision Region (264:6) to (265:50)
|
| Number of Conditions: 4
| Condition C1 --> (264:7)
| Condition C2 --> (264:34)
| Condition C3 --> (265:7)
| Condition C4 --> (265:36)
|
| Executed MC/DC Test Vectors:
|
| C1, C2, C3, C4 Result
| 1 { F, -, T, F = F }
| 2 { T, F, F, - = F }
| 3 { F, -, T, T = T }
| 4 { T, T, -, - = T }
|
| C1-Pair: covered: (1,4)
| C2-Pair: covered: (2,4)
| C3-Pair: not covered
| C4-Pair: covered: (1,3)
| MC/DC Coverage for Decision: 75.00%
|
------------------
The expression is in the form of (C1 && C2) || (C3 && C4)
. To cover C3
,
(C1 && C2)
must stay falseC3
is true, in order for it to independently determine the outcome, C4
must also be trueC1
, C2
and C4
in one test vector must be the same with their values in the other test vector, or we don't care about the value due to short-circuit effects (-
in LLVM notation)Therefore, test vectors 2 and 3 meet the criteria of masking MC/DC, but not unique-cause MC/DC, as C1
is different in two test vectors.
Limit in our current script in terms of this task:
Connect decisions from GCC reports and from LLVM reports purely by line number, which will miss many pairs due to their different conventions. Some made-up examples:
return (a == 2 && b > 1 && c < 100) // LLVM puts its MC/DC measure here ? 29 : // GCC puts its MC/DC measure here c;
return a // LLVM puts its MC/DC measure here && b || // GCC puts its MC/DC measure here c ;
For the same reason above we ignore lines with multiple decisions reported by either tool
Affected by all the oddity of GCC optimization as illustrated in "Reason 1: Source based"
Then we inspect all cases where:
&&
and ||
.With Linux 6.10-rc7 defconfig + KUnit + gcov, GCC produced in total 2583 *.gcda
files. We ran our script with 500 of them each time.
mm/percpu.c:1184
, drivers/ata/libata-core.c:5000
net/netlink/genetlink.c:264
kernel/time/clockevents.c:68
, arch/x86/kernel/apic/apic.c:1600
We found the net/netlink/genetlink.c:264
case ("Example 1") again. Others, after manual inspection, were either due to non-deterministic execution, or due to GCC oddity.
Such examples seem hard to find in kernel coverage reports. It very much depends on execution, and the code-under-test needs to be covered in a very specific way. Nonetheless, with example 1 and many user-space examples (much easier to get), the demonstrated idea is not affected.
These look like great examples. Will they appear in your LPC presentation?
These look like great examples. Will they appear in your LPC presentation?
Yes! It will be the focus of this one https://lpc.events/event/18/contributions/1895/
Reason 1: Source based
Example 1
https://elixir.bootlin.com/linux/v6.10-rc7/source/drivers/gpu/drm/drm_buddy.c#L114
GCC gcov report:
Confusing part(s):
if
statement (L114) is reported to have 10 branches instead of 2Clang Source-based Code Coverage report:
Example 2
https://elixir.bootlin.com/linux/v6.10-rc7/source/drivers/leds/led-triggers.c#L33
GCC gcov report:
Confusing part(s):
Clang Source-based Code Coverage report:
Example 3
https://elixir.bootlin.com/linux/v6.10-rc7/source/drivers/firmware/dmi_scan.c#L1068
GCC gcov report:
Confusing part(s):
Clang Source-based Code Coverage report:
Example 4
https://elixir.bootlin.com/linux/v6.10-rc7/source/fs/xattr.c#L30
GCC gcov report:
Confusing part(s):
Clang Source-based Code Coverage report: