This PR implements a new feature to collect test coverage data while running tests with -c / --coverage option enabled and report them in the output (either in human-friendly way or in JSON).
Approach
There are roughly two ways to implement coverage measurement: 1) collecting coverage in the interpreter side (like V8 JavaScript code coverage), and 2) collecting coverage in the userland side (like Istanbul test coverage)
For option 1, adding support for code coverage to our interpreter may result in tight coupling between the interpreter and coverage measurement, which I think would not a great idea especially when considering a fact that falco uses the interpreter not only in tester but also in simulator and console features.
For option 2, since we already have a special function to mutate an existing table in tester mode, testing.table_set, tables can be used to store coverage data.
This PR adds a new process called "instrumentation" to the preparation phase of testing: traversing an AST tree to find subroutines, statements, and branches to initialise coverage tables, and injecting testing.table_set statements to record executions of each subroutine, statement, or branch.
These injected statements will be evaluated along with original VCL code and produce coverage information for each test case.
After running all tests, each coverage information will be merged so that falco can calculate overall coverage percentages.
This PR implements a new feature to collect test coverage data while running tests with
-c
/--coverage
option enabled and report them in the output (either in human-friendly way or in JSON).Approach
There are roughly two ways to implement coverage measurement: 1) collecting coverage in the interpreter side (like V8 JavaScript code coverage), and 2) collecting coverage in the userland side (like Istanbul test coverage)
For option 1, adding support for code coverage to our interpreter may result in tight coupling between the interpreter and coverage measurement, which I think would not a great idea especially when considering a fact that falco uses the interpreter not only in tester but also in simulator and console features.
For option 2, since we already have a special function to mutate an existing table in tester mode,
testing.table_set
, tables can be used to store coverage data.This PR adds a new process called "instrumentation" to the preparation phase of testing: traversing an AST tree to find subroutines, statements, and branches to initialise coverage tables, and injecting
testing.table_set
statements to record executions of each subroutine, statement, or branch.These injected statements will be evaluated along with original VCL code and produce coverage information for each test case.
After running all tests, each coverage information will be merged so that falco can calculate overall coverage percentages.
Screenshot
Related Issues