Added runner and e2e tests for stats and coverage functionalities.

[kt218]

Added e2e tests that tests:

• stats shows 6 rows of klee run stats
• stats shows the same instr count as stdout
• cov shows coverage percentage
• cov colours the correct lines

Added runner tests that tests:

• stats returns 6 rows of klee run stats
• stats only return 1 stat since only 1 klee-out dir
• stats return the same instr as stdout
• cov returns only 1 coverage stat for 1 code file
• cov returns the correct JSON/dict format
• cov returns the same number of cov lines as code lines
• cov returns lines in ascending order

[ccadar]

@kt218 it's nice to have more tests. However, I am concerned that some of the checks depend on the LLVM version and might fail when we upgrade to a newer version. In particular, the number of LLVM instructions (33) might change. I think it would be better to remove those checks and only leave those which are more stable.

[kt218]

@ccadar I can change the e2e test so that it would take the stdout number of instructions and compare it with the value from stats. Or should this just be part of the runner test?

[ccadar]

@kt218 I think that would be a more flexible check, yes. I think some LLVM version-dependent tests are fine too, but they should be clearly documented and made obvious/easy to update when we upgrade to a newer LLVM. In this case, I think your proposal is better.

[kt218]

@ccadar I've modified the e2e stat test to compare with instr count from stdout. Should I also change the original tests to use regex instead of absolute matches to a string?

[andronat]

LGTM!

[ccadar]

Hi @kt218 great, let's merge this now. If you want, you can open a different PR to improve the old tests, and we can discuss them there.