eclipse / eclipse-collections

Eclipse Collections is a collections framework for Java with optimized data structures and a rich, functional and fluent API.
http://www.eclipse.org/collections
2.39k stars 596 forks source link

Update JMH Benchmarks and library dependencies. #1567

Closed donraab closed 3 months ago

donraab commented 3 months ago

I removed all of the extends AbstractJMHTestRunner code as IntelliJ has supported running JMH Benchmarks in the IDE for quite a while now. I added annotations to each of the benchmark classes for the number of forks, warmups, and iterations.

motlin commented 3 months ago

I added annotations to each of the benchmark classes for the number of forks, warmups, and iterations.

Where do these numbers come from? What are the defaults? Why add them at all?

donraab commented 3 months ago

I added annotations to each of the benchmark classes for the number of forks, warmups, and iterations.

Where do these numbers come from? What are the defaults? Why add them at all?

I was replicating the numbers that were previously inherited from AbstractJMHTestRunner. These numbers may have looked like defaults before, as they were kind of hidden away in the abstract class. Removing the extends and adding the annotations explicitly makes it more clear and easier to change for anyone who wants to run the benchmarks. I am thinking of copying these benchmarks to a separate repo where they can then be run with different versions of the JDK and different versions of Eclipse Collections. Right now you have to build EC to run these benchmarks, so they have limited use.

public abstract class AbstractJMHTestRunner
{
    @Test
    public void runTests() throws RunnerException
    {
        int warmupCount = this.warmUpCount();
        int runCount = this.runCount();
        Options opts = new OptionsBuilder()
                .include(".*" + this.getClass().getName() + ".*")
                .warmupTime(TimeValue.seconds(2))
                .warmupIterations(warmupCount)
                .measurementTime(TimeValue.seconds(2))
                .measurementIterations(runCount)
                .verbosity(VerboseMode.EXTRA)
                .forks(2)
                .build();

        new Runner(opts).run();
    }

    protected int runCount()
    {
        return 10;
    }

    protected int warmUpCount()
    {
        return 20;
    }
}
donraab commented 3 months ago

@motlin I left AbstractJMHTestRunner alone so these numbers would still be visible. If you are good with the changes, I will remove the class and update the PR.

donraab commented 3 months ago

@motlin Are there any changes you would suggest for this PR?