Open echoix opened 1 month ago
Sorry if it's a bit unordered, I wrote it on my phone, launching extra CI runs while writing to get more answers
I've did a little PoC for the following:
python -m slipcover --json --out cov_data/slipcover_raytrace_orig.json benchmarks/bm_raytrace.py
python benchmarks/create_bm_raytrace_slipcover.py
coverage json --data-file=cov_data/.coverage_raytrace_slipcover -o cov_data/slipcover_raytrace_created.json
python -m coverage run --include=benchmarks/bm_raytrace.py benchmarks/bm_raytrace.py
The file (benchmarks/create_bm_raytrace_slipcover.py):
from coverage import CoverageData
import json
from pathlib import Path
def main():
b = json.load(Path("cov_data/slipcover_raytrace_orig.json").open("r"))
a = CoverageData("cov_data/.coverage_raytrace_slipcover")
a.erase()
a.write()
files_before = b["files"]
print('b["files"]: ', b["files"])
line_data = {
str(Path().joinpath(file).resolve()): data["executed_lines"]
for file, data in b["files"].items()
}
print("line_data: ", line_data)
a.add_lines(line_data)
a.write()
if __name__ == "__main__":
main()
The json files are the same:
I'll try with branch coverage enable next.
I didn't end up finishing converting json file with branch coverage yesterday. I ended up that I had to convert a list of lists (pairs) (in the "executed_branches" of each file) to a collection of tuples (it could still remain a list).
But anyways, the format seemed the correct way, so it means the json output of slipcover contains enough info to recreate a coveragepy data file, that allows to fully use coverage.py's reporting facilities. And thus would allow me to remap paths using coverage.py's config file and using their commands.
I'm trying to integrate slipcover instead of a coverage.py, as coverage.py takes an already too long 4*35 minutes without coverage (test suite split into 4 CI jobs), to about 45-50+ min without coverage.py.
With slipcover, the times taken are about the same as without coverage, so it's great.
I'll have a couple questions, as I did my best to understand how this tool works, as there isn't a documentation for the project yet, and I tried to follow the couple issues beforehand.
I've been able to upload to codecov, as long as I set the output file name to coverage.json (if it is called .coverage.json, codecov tries to call coverage.py to merge files as it looks like if it were multiple files created with the parallel option, since it is like the
.coverage.*
pattern).So, knowing that there is a way to have a human, tabular report in slipcover, how is it possible to request it to be shown? The relevant function I think is print_coverage located here
https://github.com/plasma-umass/slipcover/blob/7f435c12084c847bea3fe7f186c1385eca3ac0c2/src/slipcover/slipcover.py#L96-L148
Up to now, I wasn't able to see that it in action yet.
It might be related to the logic in the function
printit
insideatexit
here:https://github.com/plasma-umass/slipcover/blob/7f435c12084c847bea3fe7f186c1385eca3ac0c2/src/slipcover/__main__.py#L202-L210
python
interpreter bypython -m slipcover
ran some coverage, but didn't save any data of it, and didn't display anything too. (Hopefully I only ran a subset of the tests instead of 35 minutes ;) )python -m slipcover --json
in order to have a file output (or as I thought it would), it only outputs a json to the terminal, it doesn't save it with a default name. There is still no file created that could be used to post-process what slipcover collected, either with slipcover or any custom-purpose tool.python -m slipcover --out coverage.json
, then a json file is saved, but no terminal output. However, that file is empty (0 bytes).python -m slipcover --json --out coverage.json
, then there is a file saved, no terminal output, and the file is valid, but didn't contain any files covered.python -m slipcover --json --out coverage.json --source .
, then there is a file saved, no terminal output, and the file is valid, and contains some files, (with missing lines reported), but no lines covered (that's on my side, as the source should be better tuned with the paths like I needed with coverage.py)Other questions, in no particular order
coverage combine
or even directly with any of their reporting options.--source
option be used multiple times, or it must be used once with comma separating the different paths?