haskell-graphql / graphql-api

Write type-safe GraphQL services in Haskell
BSD 3-Clause "New" or "Revised" License
406 stars 35 forks source link

Ratchet test coverage #171

Closed jml closed 6 years ago

jml commented 6 years ago

Adds a Python script to compare coverage data before and after.

Why not use hpc-coveralls? Because it doesn't work with Stack (https://github.com/guillaume-nargeot/hpc-coveralls/issues/47), and it doesn't work with multiple packages (https://github.com/guillaume-nargeot/hpc-coveralls/issues/51). Since the main thing I want is a ratchet, it's easiest just to write a script.

Generated coverage report

jml commented 6 years ago

For reasons I don't understand, I get a different report locally than we get on CI.

Local report:

$ STACK_YAML=stack-8.2.yaml stack hpc report --all >/dev/null
Getting project config file from STACK_YAML environment
Generating combined report
 57% expressions used (2172/3801)
 47% boolean coverage (9/19)
      38% guards (5/13), 4 always True, 4 unevaluated
      75% 'if' conditions (3/4), 1 unevaluated
      50% qualifiers (1/2), 1 always True
 45% alternatives used (156/344)
 81% local declarations used (70/86)
 33% top-level declarations used (348/1052)
The combined report is available at /Users/jml/src/graphql-api/.stack-work/install/x86_64-osx/lts-10.4/8.2.2/hpc/combined/custom/hpc_index.html

CI

Generating unified report
 57% expressions used (2159/3778)
 47% boolean coverage (9/19)
      38% guards (5/13), 4 always True, 4 unevaluated
      75% 'if' conditions (3/4), 1 unevaluated
      50% qualifiers (1/2), 1 always True
 45% alternatives used (152/335)
 81% local declarations used (70/86)
 32% top-level declarations used (349/1074)
The unified report is available at /root/project/.stack-work/install/x86_64-linux/lts-10.4/8.2.2/hpc/combined/all/hpc_index.html

Will investigate later.