mfridman / tparse

CLI tool for summarizing go test output. Pipe friendly. CI/CD friendly.
MIT License
998 stars 22 forks source link

Partial support for `-compare` #101

Closed mfridman closed 1 year ago

mfridman commented 1 year ago

This PR adds an experimental flag -compare to compare the current go test run against a previous one.

See #92 for more details, but the brief summary is:

You can run go test ... -json > output.json, store the JSON output, and then do another run to compare against the previous one.

A contrived example is to look at Go standard library cover changes between Go versions for select packages, for example:

Go 1.21.0 compared against 1.18.0

$ PACKAGES='fmt strings bytes bufio crypto log mime sort time'
$ go1.21 test -count=1 `echo $PACKAGES` -json -cover | tparse -compare go1.18.json

┌───────────────────────────────────────────────────────────────────┐
│  STATUS │ ELAPSED │ PACKAGE │     COVER     │ PASS │ FAIL │ SKIP  │
│─────────┼─────────┼─────────┼───────────────┼──────┼──────┼───────│
│  PASS   │  0.62s  │ bufio   │ 93.5% (+0.2%) │  87  │  0   │  0    │
│  PASS   │  2.69s  │ bytes   │ 95.4% (-0.2%) │ 146  │  0   │  0    │
│  PASS   │  0.59s  │ crypto  │  5.9% (0.0%)  │  5   │  0   │  0    │
│  PASS   │  0.39s  │ fmt     │ 95.2% (0.0%)  │  79  │  0   │  1    │
│  PASS   │  0.77s  │ log     │ 67.9% (-0.1%) │  11  │  0   │  0    │
│  PASS   │  0.90s  │ mime    │ 94.0% (+0.2%) │  24  │  0   │  0    │
│  PASS   │  2.96s  │ sort    │ 58.7% (-2.1%) │  76  │  0   │  1    │
│  PASS   │  2.21s  │ strings │ 97.6% (-0.5%) │ 122  │  0   │  0    │
│  PASS   │  8.68s  │ time    │ 92.5% (+0.7%) │ 349  │  0   │  1    │
└───────────────────────────────────────────────────────────────────┘

Going to merge this as-is and continue to chip away at refactoring the underlying logic/colorization.