adoptium / bumblebench

A microbenchmarking test framework for Eclipse Adoptium
Apache License 2.0
22 stars 30 forks source link

FormatFlagsConversionMismatchException when benchmark class name is exactly 5 characters long #34

Open dylanjtuttle opened 1 month ago

dylanjtuttle commented 1 month ago

When running a microbenchmark with a benchmark class name of 5 characters, the following runtime exception occurs:

Exception in thread "main" java.lang.RuntimeException: java.util.FormatFlagsConversionMismatchException: Conversion = s, Flags = 0
        at net.adoptopenjdk.bumblebench.core.Launcher.runBumbleMainOn(Launcher.java:64)
        at net.adoptopenjdk.bumblebench.core.Launcher.main(Launcher.java:55)
Caused by: java.util.FormatFlagsConversionMismatchException: Conversion = s, Flags = 0
        at java.base/java.util.Formatter$FormatSpecifier.failMismatch(Formatter.java:4511)
        at java.base/java.util.Formatter$FormatSpecifier.checkBadFlags(Formatter.java:3276)
        at java.base/java.util.Formatter$FormatSpecifier.checkGeneral(Formatter.java:3235)
        at java.base/java.util.Formatter$FormatSpecifier.<init>(Formatter.java:2999)
        at java.base/java.util.Formatter.parse(Formatter.java:2849)
        at java.base/java.util.Formatter.format(Formatter.java:2774)
        at java.base/java.util.Formatter.format(Formatter.java:2728)
        at java.base/java.lang.String.format(String.java:5162)
        at net.adoptopenjdk.bumblebench.core.BumbleBench.run(BumbleBench.java:266)
        at net.adoptopenjdk.bumblebench.core.MicroBench.run(MicroBench.java:107)
        at net.adoptopenjdk.bumblebench.core.BumbleBench.bumbleMain(BumbleBench.java:287)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:586)
        at net.adoptopenjdk.bumblebench.core.Launcher.runBumbleMainOn(Launcher.java:61)
        ... 1 more

This exception occurs in BumbleBench.run(), where the lines reporting the score and uncertainty at the end of the run are constructed and printed.

In order to ensure an aesthetic alignment of the two lines, a number of spaces are sometimes added to the beginning of the second line, as in this example:

  TrigBench score: 24703088.000000 (24.70M 1702.2%)
      uncertainty:   1.9%

However, the line of code which produces the correct number of spaces assumes that the name of the benchmark is at least 6 characters:

String spaces = String.format("%" + (_name.length()-5) + "s", "");

Since %0s is not a valid string format specifier, if the class name is 5 characters long, we get a crash. Although obviously less significant, if the class name is 4 characters or less, we don't get a crash, but we also don't get the alignment that this code intends to produce. For example, here is the output for a benchmark with a 4 character name, Test:

  Test score: 7955413.000000 (7.955M 1588.9%)
   uncertainty:   1.1%

The following simple benchmark can reproduce this crash:

// Clone.java
package net.adoptopenjdk.bumblebench.stringcoding;

import net.adoptopenjdk.bumblebench.core.MicroBench;

public final class Clone extends MicroBench {

    protected long doBatch(long numIterations) throws InterruptedException {
        for (long i = 0; i < numIterations; i++) {}

        return numIterations;
    }

}
dylanjtuttle commented 1 month ago

The following code would fix the crash and also ensure both lines are properly aligned no matter the length of the benchmark name. It can probably be cleaned up further, but I thought I would include it just as a starting point:

// In net/adoptopenjdk/bumblebench/core/BumbleBench.java
        if (verify()) {
            String line1_prefix = "  " + _name + " score: ";
            String line2_prefix = "  uncertainty: ";
            if (line1_prefix.length() < line2_prefix.length()) {
                line1_prefix = " ".repeat(line2_prefix.length() - line1_prefix.length()) + line1_prefix;
            } else if (line1_prefix.length() > line2_prefix.length()) {
                line2_prefix = " ".repeat(line1_prefix.length() - line2_prefix.length()) + line2_prefix;
            }
            out().println("\n" + line1_prefix + String.format("%f",_maxPeak) + " (" + score(_maxPeak) + " " + logPoints(_maxPeak) + "%)");
            out().println(line2_prefix + percentage(_uncertainty) + "%");
        } else {
            out().println("ERROR: failed verification.");
        }
karianna commented 1 month ago

@dylanjtuttle Pull Request would be welcome!