[QUESTION] Get result of image comparison using Java

[jeffradom]

I've carefully checked your demo sample code but it's not clear how programmatically check if the result of comparison true or false? Please, let me know.

Thanks a lot in advance

Jeff

[romankh3]

hello @jeffradom. The result of the comparison is ImageComparisonResult, which comtains ImageComparisonState. This is an enum, which has MATCH, MISMATCH, SIZE_MISMATCH values.

    /**
     * Result state of the comparison, where mismatch of the image sizes.
     */
    SIZE_MISMATCH,

    /**
     * Result state of the comparison, where mismatch of the images.
     */
    MISMATCH,

    /**
     * Result state of the images, where images are equal, e.g. match.
     */
    MATCH

Am I answered your question?

Best regards, Roman.

[jeffradom]

Great, that helps. BTW, what does SIZE_MISMATCH mean? In what units? Now how can I set exclusion rectangles? What measures can I use (mm, pixels, etc)? Can I somehow use your tool to compare the same web pages but in different languages for localization testing automation? I wasn't able to find any code examples> Please, let me know.

Thanks a lot in advance

Jeff

[romankh3]

image-comparison checks the pixels for getting the difference. That's why if we put images with different sizes it won't work. image-comparison requires images to be the same sizes.

You can use excluded areas to exclude some parts from the comparison.

Excluded areas set as rectangles in pixels.

For example, here is the test, which shows how excluded areas work:

    @Test
    public void testShouldIgnoreExcludedArea() {
        //given
        BufferedImage expected = readImageFromResources("expected#17.png");
        BufferedImage actual = readImageFromResources("actualMaskedComparison#58.png");
        List<Rectangle> excludedAreas = new ArrayList<>();
        excludedAreas.add(new Rectangle(131, 0, 224, 224));
        ImageComparison imageComparison = new ImageComparison(expected, actual).setExcludedAreas(excludedAreas);

        //when
        ImageComparisonResult result = imageComparison.compareImages();

        //then
        assertEquals(result.getImageComparisonState(), MATCH);
    }

all the other tests, which are showing how to use image-comparison can be found here.

I created a new GitHub issue for adding a wiki-page about setting excluded areas.

Best regards, Roman.

[jeffradom]

I see. So, ist it like Rectangle(x1, y1, x2, y2)? Can I have several exclusion rectangles? Can you suggest any Mac OS-based tools to get those measurements? BTW, I had interviewed several guys from Kharkov a few years ago when had worked for Oracle. And I myself left USSR 30 years ago and now live in San Francisco area. Was many times in Ukraine in late '80s for business Oh, I also met Epam CEO few times in Stanford Thanks

Jeff

[jeffradom]

Oh, How can I have input image files and result image file from/to a folder of my choice? Can use other than PNG formats?

[romankh3]

It’s so interesting! As we like to say that the world is not so big as can think.

Yes, you can add more than one rectangle.

I don’t know tools for it... it’s also a good point to add to wiki-page.

I tested also JPEG format. Image-comparison using BufferedImage object for comparing, so I think that it’s not so important in which format images. But I’m not sure about it.

Best regards, Roman.

[jeffradom]

Oh, How can I have input image files and result image file from/to a folder of my choice? For example abc/Resouces/image_files/ folder

[romankh3]

comparison result can be store as a png file to a folder, which is set to destination property.

here is descriptions of the properties. which are available for ImageComparison object: https://github.com/romankh3/image-comparison#configurations

Best regards, Roman.

[jeffradom]

sorry, i didn't get it. Can input files be located for example in "abc/Resouces/image_files/ folder". Please, clarify.

Thanks

Jeff

[romankh3]

That's ok. Here is an example of using destination for comparing:

    @Test
    public void testDestinationGetting() {
        //given
        BufferedImage expected = readImageFromResources("expected.png");
        BufferedImage actual = readImageFromResources("actual.png");

        //when
        ImageComparison imageComparison = new ImageComparison(expected, actual)
                .setDestination(new File("result.png"));

        //then
        assertTrue(imageComparison.getDestination().isPresent());
    }

as a result, result.png would save in the root test folder. Also, you can specify relative or absolute path for it. In destination, you need to add an image name, too.

Best regards, Roman.

[jeffradom]

I meant something like BufferedImage expected = readImageFromResources("abc/xyz/expected.png");

[romankh3]

I got it. Of course, you can. You wrote the right way to do it. BTW, it doesn't matter how you create BufferedImage object. You can use the wrapper from ImageComparisonUtil or use directly ImageIO.read() method instead.

Can you describe your use-case? I'd like to know about it.

Best regards, Roman.

[jeffradom]

Cool. I'll try and let you know in case I'll have an issue.

Thanks a lot again

Jeff

[jeffradom]

Cool. It has worked. Now I did this setPixelToleranceLevel(0.9). Does it mean that if I have 2 images very different then the comparison will return me MATCH because it din't. Please, advise

Thanks

Jeff

[romankh3]

yeah, you're right. 0.9 means, if two images have less than 90% difference, the result would MATCH.

By default, it's 0.1, which means, that two images must be more than 10% difference between them.

Best regards, Roman

[jeffradom]

well, I did set it to 90% but still see MISMATCH. It doesn't seem correct to me

[jeffradom]

this is my code //Create ImageComparison object for it. ImageComparison imageComparison = new ImageComparison( expectedImage, actualImage, resultDestination ) .setExcludedAreas(excludedAreas) .setDrawExcludedRectangles(true) .setRectangleLineWidth(3) .setPixelToleranceLevel(0.9);

[jeffradom]

My Images are

[jeffradom]

and the settings above returns MICMATCH. I'd expect MATCH

[jeffradom]

this might be a bug but I'm not sure if I might miss something though

[romankh3]

are you sure, that you properly add excluded areas? I think excluded areas right point. I'd like you to remove .setPixelToleranceLevel(0.9); it won't help you with your goal.

[romankh3]

Based on your data, I've written test for it:

    @Test
    public void shouldProperlyCompare171issue() {
        //given
        BufferedImage actual = readImageFromResources("actual#171.png");
        BufferedImage expected = readImageFromResources("expected#171.png");

        BufferedImage expectedResultImage = readImageFromResources("result#171.png");

        ImageComparison imageComparison = new ImageComparison(expected, actual)
                .setExcludedAreas(singletonList(new Rectangle(325, 50, 650, 80)))
                .setDrawExcludedRectangles(true)
                .setRectangleLineWidth(3);

        //when
        ImageComparisonResult imageComparisonResult = imageComparison.compareImages();

        //then
        assertEquals(MATCH, imageComparisonResult.getImageComparisonState());
        assertImagesEqual(expectedResultImage, imageComparisonResult.getResult());
    }

it works as expected.

the result image:

I added it to the development branch, so you can touch it and figure out why your case doesn't work.

Here is the link to the test.

[jradom]

Hi Roman,

Actually, I meant how to set comparison precision like 30% in my case and make images MATCH without any exclusions(like only few pixels differences). Please, advise.

Thanks

Jeff

[romankh3]

I don't think that it could be performed by existing options. Pixel tolerance sets the level of difference for all the pixels. the library doesn't have the ability to skip first n pixels.

You can use the options minimalRectangleSize.

Best regards, Roman.

[jradom]

I think, you didn't get it. What if I want compare images but want to make both MATCH if difference are very minor (only 10% or less). Is it possible? Otherwise you'll have false negatives/positives?

[jradom]

and I don't want exclude anything for now

[jradom]

minimalRectangleSize

I'm not sure how can I use the above property for my as a workaround? I really need a kinda way to make MATCH in case of few pixels difference. Any ideas?

[romankh3]

For now I don’t have any ideas.

[jeffradom]

I have experimented with .setPixelToleranceLevel(0.5) and that allowed me to make slightly different images to MATCH. But I didn't really understand what setPixelToleranceLevel means and how it works. Please, explain

Thanks

Jeff

[romankh3]

image-comparison compares each pixel of the images and in this comparison using pixelToleranceLevel for each pixel.

Best regards, Roman

[jradom]

I see but what will .setPixelToleranceLevel(0.5) mean and why unmatched before images started to match. If instead 0of 0.r )'ll put 0.4 slightly different images will return NOMATCH. Why? I'm really curious

Thanks

Jeff

[romankh3]

I got it. 0.4 =>must more than 40% difference of the pixels to set MISMATCH 0.5 =>must more than 50% difference of the pixels to set MISMATCH

the higher pixelToleranceLevel, the bigger pixels match.

It means, that if pixelTolarenceLevel equals 1, then all the pixels are matched.

Best regards, Roman.

[jradom]

that means it might be a bug then

[jradom]

[jradom]

the above is the expected image file

[jradom]

[jradom]

I doubt that difference ir 50% + And what do you think? I think it's 1% max difference

[romankh3]

Maybe my previous explanation wasn't so clear or I used in the wrong way English.

Let's do it again: we have two images with the same sizes. the algorithm gets the first pixel from image1 and the first pixel from image2. Let's name them pixelImage1 and pixelImage2.

PixelToleranceLevel using in comparing pixelImage1 and pixelImage2. PixelTolaranceLevel doesn't use for the whole count of difference pixels. It's using for each comparison between each pixel from image1 and pixel from image2.

Does it make sense?

Best regards, Roman.

[jeffradom]

In other words, pixelTolarenceLevel has nothing to do with the precision of the comparison set, correct? And there is no way to make what I try to achieve and no workaround, correct? I'm using a similar library in terms of PDF file comparison and it has a way to set a threshold/precision way. That way I can eliminate minor possible differences in PDF files. A developer in Germany told me that he actually behind the scenes converts PDFs into PNG files and analyze them in comparison

Thanks

Jeff

[romankh3]

I got your point. Without any updates, I don't know how to do it. It can be added, but I want to know use cases.

You can create a feature request issue with the description of the new feature, use cases where it can be helpful and I can add to the new release 4.2.0.

Best regards, Roman.

[MrMisterG]

Hello, I think it is a good idea to have a counter for "found different pixels" and to define the accuracy of the image comparison in that way, by providing a percentage of allowed different pixels. So if the image has in total 1000000 pixels, and the accuracy is set to 1%, then the result will be MATCH even if 9999 pixels are different. best, Stuart

[bestworkerr]

And my 2 cents.

I agree with @jeffradom and @MrMisterG, It could solve these cases and make library more options.

Also, due to this topic, I think, that documentation must be provided, too.

[jeffradom]

I think that the above new feature should work exactly as @MrMisterG has described

[jeffradom]

when you'll have a chance to add this? I'm very interested in the order we can use it. Please, let me know

Thanks a lot in advance

Jeff

[romankh3]

I'll do it. I need some time for implementing it

[jeffradom]

Cool. I Will patiently waiting... Please give me a buzz when done. Oh, do you want a link to PDFcompare library I had mentioned earlier? Please, let me know

Thanks again

Jeff

[romankh3]

@jeffradom, @jradom, @MrMisterG, @bestworkerr - the feature(#175) has implemented.

It could take some time to publish a new version to Maven Central.

the new version is 4.2.0

new configuration variable: allowingPercentOfDifferentPixels. Description: The percent of allowing pixels to be different to stay MATCH for comparison. E.g. percent of the pixels, which would ignore in comparison. Value can be from 0.0 to 100.00

test of the usage:

    @Test
    public void shouldAllowLessThanOnePercentDifference() {
        //given
        ImageComparison imageComparison = new ImageComparison("expected.png", "actual.png")
                .setAllowingPercentOfDifferentPixels(1);

        //when
        ImageComparisonResult result = imageComparison.compareImages();

        //then
        assertEquals(MATCH, result.getImageComparisonState());

        //when
        result = imageComparison.setAllowingPercentOfDifferentPixels(0).compareImages();

        //then
        assertEquals(MISMATCH, result.getImageComparisonState());
    }

if you have any questions - don't hesitate, ask them.

Best regards, Roman.

[jeffradom]

Hi Roman,

man, you rock :) I'll test it once it'll be available. I greatly appreciate your quick response.

Thanks a lot (Spasibo/Dyakuyu :) )

Jeff

[jeffradom]

Hi Roman,

I did check your new feature which was available in Maven library. It did work great with sample images I have. But I'll need to test that with real screenshots of our app pages. BTW, how can I if possible to borderlines of unmatched sections other than Red? Please, let me know

Tanks

Jeff

[romankh3]

For now colors for rectangles are hard coded:

• red for unmatched areas
• green for excluded areas

It was logically for me. Why do you need it?