search

kazurayam / Materials

A Groovy library to support WebDriver-based automated testing in Katalon Studio. 'Material' means a type of files created by tests runtime; such as Screenshot images etc. A directory named 'Materials' is created where files are stored with controlled path. Easy API to get access to files are provided. A convenient HTML viewer is generated.
GNU General Public License v3.0
0 stars 1 forks source link

want to display Description of each web pages in the index.html and other reports #46

Closed kazurayam closed 4 years ago

kazurayam commented 4 years ago

derived from the issue of VisualTestingInKatalonStudio : https://github.com/kazurayam/VisualTestingInKatalonStudio/issues/31

What is given

  1. in a Test Case which visits a URL and saves a screenshot with a MaterialDescription object as a parameter like this:

    Path png2 = mr.resolveScreenshotPathByURLPathComponents(
GlobalVariable[MGV.CURRENT_TESTCASE_ID.getName()],
'screenshots',
new URL(WebUI.getUrl()),
0,
'top.png',
new MaterialDescription("1", "Login page"))    // <== put a description here

    in https://github.com/kazurayam/VisualTestingInKatalonStudio/commit/6a7fc031548b9e44ca01e11ce790cb0834b9702e

  2. then the information passed as a MaterialDescription is stored in the Materials/CURA.chronos_capture/<TExecutionProfile>/<TSuiteTimestamp>/<TCaseName>/material-metadata-bundle.json file like

    {
"MaterialMetadataBundle": [
    ...
    {
        "MaterialMetadata": {
            "MaterialPath": "CURA.chronos_capture/CURA_DevelopmentEnv/20200729_130109/CURA.visitSite/screenshots/profile.php%23login.png",
            "TCaseName": "Test Cases/CURA/visitSite",
            "MaterialDescription": {
                "category": "1",
                "description": "Login page"
            },
            "InvokedMethodName": "resolveScreenshotPathByUrlPathComponents",
            "SubPath": "screenshots",
            "URL": "https://katalon-demo-cura.herokuapp.com/profile.php#login"
        }
    },

What I need to do

  1. In Materials/URA.chronos_exam/<TExecutionProfile>/<TSuiteTimestamp>/<TCaseName>/comparison-result-bundle.json file, I want to add MaterialDescription inserted into ComparisonResult node:

    {
        "ComparisonResult": {
            "expectedMaterial": {
                "Material": {
                    "url": "null",
                    "suffix": "",
                    "fileType": {
                        "FileType": {
                            "extension": "png",
                            "mimeTypes": [
                                "image/png"
                            ]
                        }
                    },
                    "path": "/Users/kazuakiurayama/katalon-workspace/VisualTestingInKatalonStudio/Materials/CURA.chronos_capture/CURA_DevelopmentEnv/20200729_130109/CURA.visitSite/screenshots/profile.php%23login.png",
                    "hrefRelativeToRepositoryRoot": "CURA.chronos_capture/CURA_DevelopmentEnv/20200729_130109/CURA.visitSite/screenshots/profile.php%23login.png",
                    "lastModified": "2020-07-29T04:02:06",
                    "description": "20200729_130109"
                }
            },
            "actualMaterial": {
                "Material": {
                    "url": "null",
                    "suffix": "",
                    "fileType": {
                        "FileType": {
                            "extension": "png",
                            "mimeTypes": [
                                "image/png"
                            ]
                        }
                    },
                    "path": "/Users/kazuakiurayama/katalon-workspace/VisualTestingInKatalonStudio/Materials/CURA.chronos_capture/CURA_DevelopmentEnv/20200729_133010/CURA.visitSite/screenshots/profile.php%23login.png",
                    "hrefRelativeToRepositoryRoot": "CURA.chronos_capture/CURA_DevelopmentEnv/20200729_133010/CURA.visitSite/screenshots/profile.php%23login.png",
                    "lastModified": "null"
                }
            },
            "diffMaterial": {
                "Material": {
                    "path": "/Users/kazuakiurayama/katalon-workspace/VisualTestingInKatalonStudio/Materials/CURA.chronos_exam/CURA_DevelopmentEnv/20200729_133011/CURA.ImageDiff_chronos/CURA.visitSite/screenshots/profile.php%23login(100.00)FAILED.png",
                    "hrefRelativeToRepositoryRoot": "CURA.chronos_exam/CURA_DevelopmentEnv/20200729_133011/CURA.ImageDiff_chronos/CURA.visitSite/screenshots/profile.php%23login(100.00)FAILED.png"
                }
            },
            "criteriaPercentage": 0.0,
            "imagesAreSimilar": false,
            "diffRatio": 100.0,
            "MaterialDescription": {
                 "category": "1.0",
                 "description": "Login page"
            }
        }
    },

  2. then I can add reporting programs to refer to the "MaterialDescription" in the comparison-result-bundle.json.

What is difficult?

In the Materials-0.79.1, the following 2 files are NOT internally linked at all.

But I need to find out how to.

kazurayam commented 4 years ago

The key is the MaterialPairs interface. It should implement MaterialMetadataBundle getActualMaterialMetadataBundle() method.

package com.kazurayam.materials

interface MaterialPairs {

    MaterialMetadataBundle getExpectedMaterialMetadataBundle()

    MaterialMetadataBundle getActualMaterialMetadataBundle()
kazurayam commented 4 years ago

Materials-0.80.0 now creates a comparison-result-bundle.json file with MaterialDescription in each ComparisonResult node.

comparison-result-bundle.json.txt

{
    "ComparisonResultBundle": [
        {
            "ComparisonResult": {
                "expectedMaterial": {
                    "Material": {
                        "url": "null",
                        "suffix": "",
                        "fileType": {
                            "FileType": {
                                "extension": "png",
                                "mimeTypes": [
                                    "image/png"
                                ]
                            }
                        },
                        "path": "/Users/kazuakiurayama/katalon-workspace/VisualTestingInKatalonStudio/Materials/CURA.chronos_capture/CURA_DevelopmentEnv/20200731_073605/CURA.visitSite/screenshots/appointment.php%23summary.png",
                        "hrefRelativeToRepositoryRoot": "CURA.chronos_capture/CURA_DevelopmentEnv/20200731_073605/CURA.visitSite/screenshots/appointment.php%23summary.png",
                        "lastModified": "2020-07-30T22:37:07",
                        "description": "20200731_073605"
                    }
                },
                "actualMaterial": {
                    "Material": {
                        "url": "null",
                        "suffix": "",
                        "fileType": {
                            "FileType": {
                                "extension": "png",
                                "mimeTypes": [
                                    "image/png"
                                ]
                            }
                        },
                        "path": "/Users/kazuakiurayama/katalon-workspace/VisualTestingInKatalonStudio/Materials/CURA.chronos_capture/CURA_DevelopmentEnv/20200731_074106/CURA.visitSite/screenshots/appointment.php%23summary.png",
                        "hrefRelativeToRepositoryRoot": "CURA.chronos_capture/CURA_DevelopmentEnv/20200731_074106/CURA.visitSite/screenshots/appointment.php%23summary.png",
                        "lastModified": "2020-07-30T22:42:14",
                        "description": "20200731_074106"
                    }
                },
                "diffMaterial": {
                    "Material": {
                        "path": "/Users/kazuakiurayama/katalon-workspace/VisualTestingInKatalonStudio/Materials/CURA.chronos_exam/CURA_DevelopmentEnv/20200731_074107/CURA.ImageDiff_chronos/CURA.visitSite/screenshots/appointment.php%23summary(0.00).png",
                        "hrefRelativeToRepositoryRoot": "CURA.chronos_exam/CURA_DevelopmentEnv/20200731_074107/CURA.ImageDiff_chronos/CURA.visitSite/screenshots/appointment.php%23summary(0.00).png"
                    }
                },
                "criteriaPercentage": 1.43,
                "imagesAreSimilar": true,
                "diffRatio": 0.0,
                "MaterialDescription": {
                    "category": "1",
                    "description": "Appointment Confirmation page"
                }
            }
        },
kazurayam commented 4 years ago

tagged 0.80.0 and will release it.

kazurayam commented 4 years ago

I checked how much work will be involved to change the Materials/index.html to display MaterialDescription. I found it would require more work than I expected. The com.kazurayam.materials.view.IndexerBase program is not aware of the comparison-result-bundle.json file at all. And it seems that it is difficult to change IndexerBase. The IndexerBase scans the directory tree as is without know which file is what.

kazurayam commented 4 years ago

Still I can improve the com.kazurayam.visualtesting.ImageDiffsLister so that it emits reports with MaterialDescription information of each URL. The reports include:

  1. Materials/misc/imageDiffsList.csv
  2. Materials/misc/imageDiffsList.md
  3. Materials/misc/imageDiffsList.ps1