search

kazurayam / inspectus

The Inspectus library enables automated UI tests in Java/Groovy to perform "Visual Inspection" on top of the "materialstore"
Apache License 2.0
0 stars 0 forks source link

Why all of Metadata attributes are painted green? #108

Closed kazurayam closed 9 months ago

kazurayam commented 9 months ago

derived from the https://github.com/kazurayam/materialstore/issues/450

I created the src/test/java/issues/issue108/AppleTwinsDiffTest.java

It produced the following output:

Screenshot 2023-12-09 at 18 44 00

Why all attributes are painted green?

kazurayam commented 9 months ago

The background color of green --- what does it mean?

Screenshot 2023-12-09 at 20 35 07

<span class="matched-value">"resized"</span> is painted green.

But the <span>Production</span> and <span>Development</span> have un-matching values but both of them are painted green. That's simply wrong.

kazurayam commented 9 months ago

At https://github.com/kazurayam/inspectus/tree/issue108-print-mpg, I can execute src/test/java/issues/issue108/AppleTwinsDiffTest.java.

When I execute it, I got the following report.

Screenshot 2023-12-09 at 21 21 40

All of Metadata atrributes has "matchedByAster": true, which is inappropriate.

At which timing the "matchedByAster" is set to be true?

kazurayam commented 9 months ago

In the materialstore project, derived an issue : com.kazurayam.materialstore.core.metadata.MetadataImpl#annotate() method is NOT tested

kazurayam commented 9 months ago

issues.issue108.AppleTwinsDiffTest class emitted:

Screenshot 2023-12-10 at 5 50 45

I noticed that the Zipper class reported that all Materials, all Metadata attributes has "matchedByAster":true:

[Test worker] DEBUG com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials left Y c33b307 png {"description":{"key":"description","value":"original","matchedByAster":true}, "environment":{"key":"environment","value":"Production","matchedByAster":true}, "image-height":{"key":"image-height","value":"199","matchedByAster":true}, "image-width":{"key":"image-width","value":"200","matchedByAster":true}, "step":{"key":"step","value":"01","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"kazurayam.github.io","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/myApple/page1.html","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}}

I have got an idea why this happens.

kazurayam commented 9 months ago

At the https://github.com/kazurayam/inspectus/tree/issue108-print-left-and-right-materiallist you can reproduce this.

I executed the issues.issue108.AppleTwinsDiffTest ant got this output in the Standard Error:

SLF4J: Class path contains multiple SLF4J providers.
SLF4J: Found provider [org.slf4j.simple.SimpleServiceProvider@14982a82]
SLF4J: Found provider [ch.qos.logback.classic.spi.LogbackServiceProvider@4ee5b2d9]
SLF4J: See https://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual provider is of type [org.slf4j.simple.SimpleServiceProvider@14982a82]
[Test worker] INFO io.github.bonigarcia.wdm.WebDriverManager - Using chromedriver 120.0.6099.71 (resolved driver for Chrome 120)
[Test worker] INFO io.github.bonigarcia.wdm.WebDriverManager - Exporting webdriver.chrome.driver as /Users/kazuakiurayama/.cache/selenium/chromedriver/mac64/120.0.6099.71/chromedriver
Starting ChromeDriver 120.0.6099.71 (9729082fe6174c0a371fc66501f5efc5d69d3d2b-refs/branch-heads/6099_56@{#13}) on port 29315
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Dec 10, 2023 6:10:36 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find CDP implementation matching 120
Dec 10, 2023 6:10:36 AM org.openqa.selenium.chromium.ChromiumDriver lambda$new$3
WARNING: Unable to find version of CDP to use for . You may need to include a dependency on a specific version of the CDP using something similar to `org.seleniumhq.selenium:selenium-devtools-v86:4.6.0` where the version ("v86") matches the version of the chromium-based browser you're using and the version number of the artifact is the same as Selenium's.
[Test worker] WARN com.kazurayam.inspectus.core.internal.AbstractService - backup is not specified in the parameters. will skip restoring the previous JobTimestamp directories from the backup store.
[Test worker] DEBUG com.kazurayam.inspectus.core.internal.TwinsDiff - llllllllllllll MaterialList left llllllllllllllllllll
{
  "jobName": "myApple",
  "jobTimestamp": "20231210_061036",
  "queryOnMetadata": {
    "*": "re:.*"
  },
  "materialList": [
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061036",
      "id": "c33b3076315c1b288bb186627e4365c36c64e0f1",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "original",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Production",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "199",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "200",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "01",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple/page1.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"original\", \"environment\":\"Production\", \"image-height\":\"199\", \"image-width\":\"200\", \"step\":\"01\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple/page1.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple/page1.html",
      "relativeUrl": "myApple/20231210_061036/objects/c33b3076315c1b288bb186627e4365c36c64e0f1.png",
      "diffability": "AS_IMAGE"
    },
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061036",
      "id": "7481bdecd834029ed4f220c01e1e8d9bc904f0b9",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "resized",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Production",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "100",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "100",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "02",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple/page2.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"resized\", \"environment\":\"Production\", \"image-height\":\"100\", \"image-width\":\"100\", \"step\":\"02\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple/page2.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple/page2.html",
      "relativeUrl": "myApple/20231210_061036/objects/7481bdecd834029ed4f220c01e1e8d9bc904f0b9.png",
      "diffability": "AS_IMAGE"
    },
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061036",
      "id": "5cd8ae1b89721741bf00a6ef29b421a3c06e310e",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "lorem ipsum",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Production",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "199",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "200",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "03",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple/page3.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"lorem ipsum\", \"environment\":\"Production\", \"image-height\":\"199\", \"image-width\":\"200\", \"step\":\"03\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple/page3.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple/page3.html",
      "relativeUrl": "myApple/20231210_061036/objects/5cd8ae1b89721741bf00a6ef29b421a3c06e310e.png",
      "diffability": "AS_IMAGE"
    },
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061036",
      "id": "615e7267146e307a390756a2bb0f8fbbf767a682",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "rotated",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Production",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "199",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "200",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "04",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple/page4.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"rotated\", \"environment\":\"Production\", \"image-height\":\"199\", \"image-width\":\"200\", \"step\":\"04\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple/page4.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple/page4.html",
      "relativeUrl": "myApple/20231210_061036/objects/615e7267146e307a390756a2bb0f8fbbf767a682.png",
      "diffability": "AS_IMAGE"
    }
  ]
}llllllllllllllllllll
[Test worker] DEBUG com.kazurayam.inspectus.core.internal.TwinsDiff - rrrrrrrrrrrrrr + MaterialList right rrrrrrrrrrrrrrrrrr
{
  "jobName": "myApple",
  "jobTimestamp": "20231210_061048",
  "queryOnMetadata": {
    "*": "re:.*"
  },
  "materialList": [
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061048",
      "id": "c33b3076315c1b288bb186627e4365c36c64e0f1",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "original",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Development",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "199",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "200",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "01",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple-alt/page1.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"original\", \"environment\":\"Development\", \"image-height\":\"199\", \"image-width\":\"200\", \"step\":\"01\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple-alt/page1.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple-alt/page1.html",
      "relativeUrl": "myApple/20231210_061048/objects/c33b3076315c1b288bb186627e4365c36c64e0f1.png",
      "diffability": "AS_IMAGE"
    },
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061048",
      "id": "c33b3076315c1b288bb186627e4365c36c64e0f1",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "resized",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Development",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "199",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "200",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "02",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple-alt/page2.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"resized\", \"environment\":\"Development\", \"image-height\":\"199\", \"image-width\":\"200\", \"step\":\"02\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple-alt/page2.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple-alt/page2.html",
      "relativeUrl": "myApple/20231210_061048/objects/c33b3076315c1b288bb186627e4365c36c64e0f1.png",
      "diffability": "AS_IMAGE"
    },
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061048",
      "id": "5cd8ae1b89721741bf00a6ef29b421a3c06e310e",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "lorem ipsum",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Development",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "199",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "200",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "03",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple-alt/page3.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"lorem ipsum\", \"environment\":\"Development\", \"image-height\":\"199\", \"image-width\":\"200\", \"step\":\"03\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple-alt/page3.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple-alt/page3.html",
      "relativeUrl": "myApple/20231210_061048/objects/5cd8ae1b89721741bf00a6ef29b421a3c06e310e.png",
      "diffability": "AS_IMAGE"
    },
    {
      "jobName": "myApple",
      "jobTimestamp": "20231210_061048",
      "id": "191586b67024f1620995238bb1fc5382647cd793",
      "fileType": "png",
      "metadata": {
        "description": {
          "key": "description",
          "value": "rotated",
          "matchedByAster": true
        },
        "environment": {
          "key": "environment",
          "value": "Development",
          "matchedByAster": true
        },
        "image-height": {
          "key": "image-height",
          "value": "199",
          "matchedByAster": true
        },
        "image-width": {
          "key": "image-width",
          "value": "200",
          "matchedByAster": true
        },
        "step": {
          "key": "step",
          "value": "04",
          "matchedByAster": true
        },
        "URL.host": {
          "key": "URL.host",
          "value": "kazurayam.github.io",
          "matchedByAster": true
        },
        "URL.path": {
          "key": "URL.path",
          "value": "/myApple-alt/page4.html",
          "matchedByAster": true
        },
        "URL.port": {
          "key": "URL.port",
          "value": "80",
          "matchedByAster": true
        },
        "URL.protocol": {
          "key": "URL.protocol",
          "value": "https",
          "matchedByAster": true
        }
      },
      "identification": "{\"description\":\"rotated\", \"environment\":\"Development\", \"image-height\":\"199\", \"image-width\":\"200\", \"step\":\"04\", \"URL.host\":\"kazurayam.github.io\", \"URL.path\":\"/myApple-alt/page4.html\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}",
      "metadataURL": "https://kazurayam.github.io/myApple-alt/page4.html",
      "relativeUrl": "myApple/20231210_061048/objects/191586b67024f1620995238bb1fc5382647cd793.png",
      "diffability": "AS_IMAGE"
    }
  ]
}rrrrrrrrrrrrrrrrrrrr

As you see, the MaterialList "left" and "right" has ""matchedByAster": true" already. Why?

kazurayam commented 9 months ago

https://github.com/kazurayam/inspectus/blob/issue108-print-left-and-right-materiallist/src/main/java/com/kazurayam/inspectus/core/internal/TwinsDiff.java at LINE#130 has this:

        try {
            // get the MaterialList of the left (Production Environment)
            MaterialList left = store.select(jobName, jobTimestampLeft, QueryOnMetadata.ANY);

            logger.debug("llllllllllllll MaterialList left llllllllllllllllllll\n" +
                    left.toJson(true) +
                    "llllllllllllllllllll");

            // get the MaterialList of the right (Development Environment)
            MaterialList right = store.select(jobName, jobTimestampRight, QueryOnMetadata.ANY);

            logger.debug("rrrrrrrrrrrrrr + MaterialList right rrrrrrrrrrrrrrrrrr\n" +
                    right.toJson(true) +
                    "rrrrrrrrrrrrrrrrrrrr");

It seems that the statement store.select(jobName, jobTimestampLeft, QueryOnMetadata.ANY) generated "matchedByAster": true for all Metadata attributes.

Let me check it.

kazurayam commented 9 months ago

https://github.com/kazurayam/inspectus/blob/issue108-print-left-and-right-materiallist/src/main/java/com/kazurayam/inspectus/core/internal/ChronosDiff.java LINE#75 has the following statement:

            MaterialList currentMaterialList = store.select(jobName, jobTimestamp);

Wow! Different! With or without QueryOnMetadata.ANY!

kazurayam commented 9 months ago

I modified the TwinsDiff to be the same as the ChronosDiff (no QueryOnMetadata.ANY), but the resulting HTML was the same. The json image of the MaterialList was the same.

So I need to look at the source of the com.kazurayam.materialstore.core.StoreImpl, atstore.select()` method.

@Override
    public MaterialList select(JobName jobName, JobTimestamp jobTimestamp)
            throws MaterialstoreException {
        Jobber jobber = this.getJobber(jobName, jobTimestamp);
        return jobber.selectMaterials(QueryOnMetadata.ANY);
    }

Here I found that QueryOnMetadata.ANY was used as default.

...

I need to see com.kazurayam.materialstore.core.Jobber, LINE#82 selectMaterials().

    public MaterialList selectMaterials(final IFileType fileType, final QueryOnMetadata query) {
        Objects.requireNonNull(query);
        Objects.requireNonNull(fileType);
        final MaterialList result = new MaterialList(jobName, jobTimestamp, query);
        for (IndexEntry indexEntry : index) {
            if (query.equals(QueryOnMetadata.ANY) || query.matches(indexEntry.getMetadata())) {
                if (fileType.equals(FileType.NULL_OBJECT) || fileType.equals(indexEntry.getFileType())) {
                    Material material = new Material(store, getJobName(), getJobTimestamp(), indexEntry);
                    result.add(material);
                }
            }
        }
        return result;
    }

I should look into this statemet.

                    result.add(material);

com.kazurayam.materialstore.core.MaterialList has this code:

    public void add(Material material) {
        material.getMetadata().annotate(query);
        materialList.add(material);
    }

And finally, com.kazurayam.materialstore.core.metadata.MetadataImple has the annotate(QueryOnMetadata) method, which acutally put the "matchdByAster": true

    @Override
    public void annotate(final QueryOnMetadata query) {
        Objects.requireNonNull(query);
        attributes.keySet().forEach( key -> {
            if (matchesByAster(query, key)) {
                attributes.get(key).setMatchedByAster(true);
            }
            if (matchesIndividually(query, key)) {
                attributes.get(key).setMatchedIndividually(true);
            }
        });
    }
kazurayam commented 9 months ago

don v0.11.2