Skip stdout which is not benchmark result, or provide better error message

sebastien-rosset commented 3 years ago

After enabling gobenchdata in a GitHub action with the out-of-the-box example yml file, the benchmark failed with the following error:

📊 Running benchmarks...
panic: cpu: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz: could not parse run: strconv.Atoi: parsing "": invalid syntax (line: cpu: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz)

goroutine 1 [running]:
main.main()
    /tmp/build/main.go:230 +0x19b3

For a minute I was wondering if this was the repo under test invoking panic, or caused by error in my github action yml file, or a bug in gobenchdata. I can see now this error is returned by this line of code: https://github.com/bobheadxi/gobenchdata/blob/45436a704a57990aa358789f44ebd133cc66f08f/bench/parser.go#L102

Instead of the following code to parse the line:

split := strings.Split(line, "\t")

How about using a regex that matches the expected benchmark output? Maybe skip lines that don't match the expected output? Define a config parameter in the yml file to decide whether to skip or not? Extra output may be generated by go test:

#go test -bench=.
goos: linux
goarch: amd64
pkg: github.com/CiscoM31/godata
cpu: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz

The cpu line above causes the panic

bobheadxi commented 3 years ago

yeah it seems the CPU data is new - I ran into this in #46 🙁 A better solution like the one you mentioned is probably needed. I do believe the benchmark package might be exported now in the standard library, I'll probably look into that too. Thanks for opening this issue!

piotrekmonko commented 2 years ago

@bobheadxi could you take a look at an approach to fixing this in #49 , bit different from #48 ?

bobheadxi / gobenchdata

Skip stdout which is not benchmark result, or provide better error message #47