metan-ucw / runltp-ng

Minimalistic LTP testrunner
11 stars 16 forks source link

Syzkaller reproducer falsely reported as failed #20

Open shunghsiyu opened 4 years ago

shunghsiyu commented 4 years ago

While inspecting the resulting file I comes across a case where the Syzkaller reproducer is marked as failed, but in reality it passed.

{
  "tid": "ae496c7eb605ca41cd6192cc678f08ad2020f1c3",
  "runs": 1,
  "warnings": 0,
  "runtime": "13.5256508011371",
  "broken": 0,
  "skipped": 0,
  "failed": 1,
  "passed": 0,
  "log": [
    "...omitted...",
    "[  573.830051] 0syzwrap  -d /opt/ltp/testcases/bin -n ae496c7eb605ca41cd6192cc678f08ad2020ff1c3; echo cmd-exit-115-$?",
    "...omitted...",
    "Summary:",
    "passed   1",
    "failed   0",
    "skipped  0",
    "warnings 0",
    "...omitted..."
  ]
}

Further inspection of the raw log file it look like the it is an unfortunate case where the output of printk got interleaved with the output of syzwrap exit status.

[  586.379945] 0cmd-exit-115-001: RBP: 00007ffcd6f1def0 R08: 0000000000000001 R09: 00007ffcd6f1def0

I think adding some delimiter around cmd-exit-115-$? might relieve this issue, but reliable retrieval of exit status we will need #19.