[Checks] Mem.BytesPerOp not supported

[josephgardner]

The following check tests for a regression in Mem.BytesPerOp. However, the result is always 0.

checks:
- name: BytesPerOp
  package: .
  benchmarks: [ '.' ]
  diff: (current.Mem.BytesPerOp - base.Mem.BytesPerOp) / base.Mem.BytesPerOp * 100
  thresholds:
    max: 25

Results:

{
  "Status": "pass",
  "Checks": {
    "BytesPerOp": {
      "Status": "pass",
      "Diffs": [
        {
          "Status": "pass",
          "Package": "...",
          "Benchmark": "...",
          "Value": 0
        }
      ],
      "Thresholds": {
        "Min": null,
        "Max": 25
      }
    }
  }
}

Sample input:

[
  {
    "Date": 1651034107,
    "Suites": [
      {
        "Goos": "linux",
        "Goarch": "amd64",
        "Pkg": "...",
        "Benchmarks": [
          {
            "Name": "...",
            "Runs": 5,
            "NsPerOp": 111754445,
            "Mem": {
              "BytesPerOp": 14284630,
              "AllocsPerOp": 503144,
              "MBPerSec": 0
            }
          }
        ]
      }
    ]
  }
]

[bobheadxi]

What is your go test command? IIRC you must enable memory profiling with -benchmem, e.g:

go test -bench . -benchmem ./... | gobenchdata --json bench.json

[josephgardner]

The "sample input" in the OP is an extract from the contents of my bench.json which does include "BytesPerOp": 14284630,

The problem is when I feed this json file to gobenchdata checks eval prev-bench.json bench.json --json checks-results.json the diff is always 0.

[bobheadxi]

Thank you for the detail, I will take a look! I misread the original issue 😅

[bobheadxi]

I have added a test case that replicates this scenario and it appears to be fixed! https://github.com/bobheadxi/gobenchdata/pull/58/files#diff-5090d56e4c47ae794394ba4372c98072a2a530a05f7c9f41588b5db0fb2710aaR53-R58

This patch is included in https://github.com/bobheadxi/gobenchdata/releases/tag/v1.0.4