tracking issue: gobenchdata v1

bobheadxi commented 4 years ago

If you use gobenchdata, I'd love to hear from you! Please feel free to respond to this issue or reach out to me at robert@bobheadxi.dev

Previews are now available! See #38 for more details

Won't Change

benchmark storage format will stay the same - all existing run histories will work with v1 without a change
- by extension, existing generated web apps will continue to work

Will Change

Improvements for CI Usage

⚠️ to continue using the gobenchdata Action as-is when I release v1, you will need to pin your action version to bobheadxi/gobenchdata@v0.5.1!

[x] https://github.com/bobheadxi/gobenchdata/issues/28 and related output (check status? ~PR comments? generate image, a la https://github.com/bobheadxi/gobenchdata/issues/32?~)
- at the moment my thinking is: all the output is available as JSON through some means, so advanced reports like PR comments and automated image generation+updates should be handled in separate actions (this one already does a lot as it is, since it includes the publishing mechanism)
[x] More generic usage: allow pushing to different branches/repos
[x] Follow GitHub Actions versioning conventions (i.e. use bobheadxi/gobenchdata@v1)

Revamped Visualization

Better charts, more control over what gets charted (chart groups, more places to put descriptions, better ways to select groups of benchmarks)

[x] https://github.com/bobheadxi/gobenchdata/issues/18
[x] https://github.com/bobheadxi/gobenchdata/issues/15
[x] investigate how multiple runs per day are rendered (https://github.com/bobheadxi/gobenchdata/issues/36#issuecomment-624363811)

CLI improvements

[x] https://github.com/bobheadxi/gobenchdata/issues/37

Drafts

updated usage

bobheadxi commented 4 years ago

/cc @bonedaddy, @JRascagneres if y'all still use this!

JRascagneres commented 4 years ago

All the changes look great. I'm quite looking forward to https://github.com/bobheadxi/gobenchdata/issues/28 - Would be nice to know we have an issue without having to visit the page frequently.

bobheadxi commented 4 years ago

I'm drafting up how I want usage to look like in the v1 README - let me know if you have any thoughts 🚀

bonedaddy commented 4 years ago

i miss almost every github notification lol, but I definitely still use it and extensively so, basically any repo that has benchmarks uses gobenchdata.

My thoughts:

Standalone image generation sounds awesome, several times I've taken screenshots to do this.
YAML based configuration is also pretty cool and would make adding new features which require configuration a lot easier to use as you wouldn't have to use verbose cli commands
One issue I've noticed is if you do a benchmark run twice in a day, both runs won't show up as they are listed by days, and I think what happens is the most recent benchmark shows. So perhaps date + time would be a useful display method? Not really an issue i guess, more so a desirable feature but not sure how applicable it is.
Noticeable performance trends would be really cool
When there's previews available I can test
Enable github sponsors so I can sponsor the repo :D

bobheadxi commented 4 years ago

previews of bobheadxi/gobenchdata@v1 are now available! see https://github.com/bobheadxi/gobenchdata/pull/38

bobheadxi commented 4 years ago

bobheadxi/gobenchdata@v1 is now available :) https://github.com/bobheadxi/gobenchdata/releases/tag/v1.0.0

bobheadxi / gobenchdata