Allow to use an existing test report instead of creating a new one

[lo1tuma]

I’m running gemini test as part of my CI pipeline in order to prevent deploying unwanted visual changes. This works quite well. I also want to use the gemini-gui to make it easy to accept intentional changes. The problem here is that I have to run the tests again in the GUI instead of reusing the existing report made by the CI pipeline.

Ideally there would be even an option to disable running tests via the GUI as this could produce a different result than the test run before that was executed by the CI pipeline (e.g. if new commits have been pushed).

[sipayRT]

The problem here is that I have to run the tests again in the GUI instead of reusing the existing report made by the CI pipeline

sorry, I don't understand how can you reuse the existing report. Is that report was generated by gemini test command?

[lo1tuma]

@sipayRT Exactly. I’m using gemini test the create a a test report which I then want to use in gemini-gui.

[SevInf]

Actually, our team will also be interested in this feature.

The way it can be possibly implemented:

• we introduce new report format, which internally can look something like this:

{
   "mySuteName": {
       "children": {
           "subSuiteName": ...
      },
      "browsers": {
            "chrome": {
                 "image": "{base64-encoded image}"
            },
            "ie": {
                 "error": "stacktrace"
             }
      }
   }
}

• gui can read this format and display the result the same way as if it was run in gui in the first place.

Then CI system can publish new json report alongside html and developers will be able to update the images without re-running the tests.

[j0tunn]

@SevInf I think this could be done via some gemini-json-reporter plugin. Also this plugin can emulate gemini events on load when started with some command line option (something like --play). gemini-gui will receive those events and display all the stuff. What do you think?

[SevInf]

@j0tunn yes, plugin might work. Does it mean we need to introduce a programmatic way to launch the gui with injected gemini instance?

[lo1tuma]

I have implemented a custom reporter which generates json but I've decided to go with a flat array instead of the tree structure suggested above.

I also added a hash value to each test result so that it can be uniquely identified. This allows you to make changes to the result e. g. via a GUI.

As a proof of concept I am currently trying to implement a separate GUI which less features than this one. It only allows you to see the test results and approving visual changes. As input / config it takes the json file from my custom reporter and the path to the reference images.

I hope to have something ready to publish soon.

[j0tunn]

Does it mean we need to introduce a programmatic way to launch the gui with injected gemini instance

No, gemini-gui and plugin will share the same gemini instance

[lo1tuma]

I’ve published the json reporter and alternative GUI.

Feedback would be very welcomed.

[SevInf]

@lo1tuma wow, awesome work! I was not able to check it our environment since our CI is still on node 4, but I like the polished looks of the UI and I'm pretty convinced that this is how main GUI and HTML report should look like. Will you be interested in merging it into mainline?

[lo1tuma]

@SevInf You could run the GUI in a docker container and simply mount the folder where the reference screenshots are located, then the node 6 dependency shouldn’t be a problem. But I think the json-reporter needs node 6 as well.

I would be definitely interested in merging this into the mainline GUI, but currently the feature set is quite different and I’m not sure if there is a good way to support both feature sets in the same GUI. Let me know if you have some good ideas about this.