search

sourcenetwork / defradb

DefraDB is a Peer-to-Peer Edge Database. It's the core data storage system for the Source Network Ecosystem, built with IPLD, LibP2P, CRDTs, and Semantic open web properties.
434 stars 42 forks source link

Benchmark CI is broken that outputs the result on a PR #1062

Open shahzadlone opened 1 year ago

shahzadlone commented 1 year ago

The benchmark results are not being properly formatted when they are dumped on a PR.

Can sort this out when we fix the benchmarking itself, until then is low priority.

shahzadlone commented 1 year ago

Issue is in this step: Run the benchmark comparisons Of this job: Run details Run the benchmarks on the started EC2 runner Of this workflow: Lint Then Benchmark Workflow

Basically the benchstat tool got updated so now we get:

Run ${GOPATH}/bin/benchstat -html -alpha 1.1 develop.txt current.txt | sed -n "/<body>/,/<\/body>/p" > comparison.html && ./tools/scripts/pretty-benchstat-html.sh comparison.html > pretty-comparison.md
  ${GOPATH}/bin/benchstat -html -alpha 1.1 develop.txt current.txt | sed -n "/<body>/,/<\/body>/p" > comparison.html && ./tools/scripts/pretty-benchstat-html.sh comparison.html > pretty-comparison.md
  shell: /usr/bin/bash -e {0}
  env:
    DEFAULT_BENCHMARK_TYPE: SHORT
    HOME: /actions-runner/_work
    GOPATH: /actions-runner/_work/go
flag provided but not defined: -html
shahzadlone commented 1 year ago

After this do: #1079

shahzadlone commented 1 year ago

Separating linter in #1434 / #1435. Once merged turning the benchmarks off until benchmarks are fixed.

shahzadlone commented 1 year ago

Closing https://github.com/sourcenetwork/defradb/issues/980 as the place remaining was in benchmark action which is now disabled. When we fix benchmarks the set-output will be done.