Open schneiderfelipe opened 3 years ago
Hey,I generally think the Julia project's benchmarking practices are inadequate to correctly evaluate the impact of code changes, in particular the focus on minimum times, and stand by my comment about being surprised if Hyperscript offers performance advantages over any package specifically written to have good performance. :)
I'm also increasingly not the best person to continue to maintain this package, as I haven't used Julia's web ecosystem in a long time. I wonder if there's a possibility of having maintainership taken over by e.g. the folks at JuliaWeb...
Hey,I generally think the Julia project's benchmarking practices are inadequate to correctly evaluate the impact of code changes, in particular the focus on minimum times,...
That's OK. I understand your point. How could I change this PR to reflect a better way of evaluating the impact of code changes?
...and stand by my comment about being surprised if Hyperscript offers performance advantages over any package specifically written to have good performance. :)
This could change in the future. #13 is an example. I'm interested in working on improving the performance of Hyperscript.jl, if you don't mind. But, for that, a way of measuring performance is required.
I'm also increasingly not the best person to continue to maintain this package, as I haven't used Julia's web ecosystem in a long time. I wonder if there's a possibility of having maintainership taken over by e.g. the folks at JuliaWeb...
@yurivish Is there someone we could talk to to make this transfer? I could send a message to #web Slack, if you don't mind.
@yurivish Is there someone we could talk to to make this transfer? I could send a message to #web Slack, if you don't mind.
Sure, thanks @schneiderfelipe.
How could I change this PR to reflect a better way of evaluating the impact of code changes?
The biggest improvement would be showing distributions instead of individual measurements. E.g. each test could instead be shown as a be a collection of tickmarks
or a violin plot visualizing the distribution of timing values for each test.
This adds a simple benchmark suite (resolves #19). A simple report can be generated as well, with graphs such as
The report can be generated with
Most of the benchmarks are very naive and would benefit from critical feedback.