smarr / ReBenchDB

ReBenchDB records benchmark results and provides customizable reporting to track and analyze run-time performance of software programs.
MIT License
13 stars 6 forks source link

Report GitHub not having commit properly #71

Open smarr opened 3 years ago

smarr commented 3 years ago

I think, ReBench will fail at the moment, when GitHub doesn't have the commit for that data.

At least, ReBenchDB will record just an HTTP error, as below. But, it would be more useful to record that properly with meta data like a time, and request infos to be able to understand what's going on.

Current details in error log:

RequestError [HttpError]: No commit found for SHA: 1609ccc8ffec6879c258110368a08e78fce95ea5
    at /home/node-deploy/sites/ReBenchDB/node_modules/@octokit/request/dist-src/fetch-wrapper.js:52:31
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:94:5)
    at sendRequestWithRetries (/home/node-deploy/sites/ReBenchDB/node_modules/@octokit/auth-app/dist-src/hook.js:65:16)
    at GitHub.postCommitComment (/home/node-deploy/sites/ReBenchDB/src/github.ts:58:20) {
  status: 422,
  headers: {
    'access-control-allow-origin': '*',
    'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset',
    connection: 'close',
    'content-length': '170',
    'content-security-policy': "default-src 'none'",
    'content-type': 'application/json; charset=utf-8',
    date: 'Wed, 20 Oct 2021 22:45:32 GMT',
    'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
    server: 'GitHub.com',
    'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
    vary: 'Accept-Encoding, Accept, X-Requested-With',
    'x-content-type-options': 'nosniff',
    'x-frame-options': 'deny',
    'x-github-media-type': 'github.v3; format=json',
    'x-github-request-id': 'DCF8:18B4:89A1431:8C6DB36:61709C0C',
    'x-ratelimit-limit': '10650',
    'x-ratelimit-remaining': '10648',
    'x-ratelimit-reset': '1634772689',
    'x-ratelimit-resource': 'core',
    'x-ratelimit-used': '2',
    'x-xss-protection': '0'
  },
  request: {
    method: 'POST',
    url: 'https://api.github.com/repos/smarr/TruffleSOM/commits/1609ccc8ffec6879c258110368a08e78fce95ea5/comments',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': 'octokit-rest.js/18.5.3 octokit-core.js/3.4.0 Node.js/15.14.0 (linux; x64)',
      authorization: 'token [REDACTED]',
      'content-type': 'application/json; charset=utf-8'
    },
    body: '{"body":"#### Performance changes for 624b658ebe0ab7a3da2161dad5f3a1848fd70e6a...1609ccc8ffec6879c258110368a08e78fce95ea5\\n\\n![Summary Over All Benchmarks](https://rebench.stefan-marr.de//static/reports/TruffleSOM-624b65-1609cc_files/overview.png)\\nSummary Over All Benchmarks\\n\\n[Full Report](https://rebench.stefan-marr.de/compare/TruffleSOM/624b658ebe0ab7a3da2161dad5f3a1848fd70e6a/1609ccc8ffec6879c258110368a08e78fce95ea5)"}',
    request: { hook: [Function: bound bound register] }
  },
  documentation_url: 'https://docs.github.com/rest/reference/repos#create-a-commit-comment'
}