effekt-lang / effekt

A language with lexical effect handlers and lightweight effect polymorphism
https://effekt-lang.org
MIT License
334 stars 24 forks source link

Improve benchmark arguments #666

Closed marvinborner closed 1 week ago

marvinborner commented 2 weeks ago

Lately much of our benchmark data has become less useful since the measurements got really small. Our JS and LLVM backends also have fewer errors like stack overflows, allowing us to use larger arguments in general.

We should still consider when and how often we want to normalize the arguments, since it makes historical data less comparable.

I've also started writing a benchmark config for reference arguments, such that we can compare the data with the benchmarks from other languages (this is not finished right now, since it's hard to find good reference values).

phischu commented 1 week ago

Can I just merge this or not?

marvinborner commented 1 week ago

Yes, I think we can just merge.