search

zinserjan / wdio-screenshot

A WebdriverIO plugin. Additional commands for taking screenshots with WebdriverIO.
MIT License
109 stars 43 forks source link

Join forces #5

Open christian-bromann opened 8 years ago

christian-bromann commented 8 years ago

Hi,

I stumbled upon this great project and wondering if you would like to join forces and merge this into the webdrivercss project?

What do you think about that?

Cheers

zinserjan commented 8 years ago

I have to think about to merge everything into webdrivercss. Currently, I think it's better to seperate webdrivercss into multiple packages as suggested in webdriverio/webdrivercss#113.

Webdrivercss does at the moment many different things:

I tried this transformation with wdio-screenshot and wdio-visual-regression-service. wdio-screenshot is just for taking screenshots, wdio-visual-regression-service is for testing of all kinds of elements in different resolutions with any kind of comparison method. At the moment there's only a local compare algorithm but it's easy to build a custom compare method. For an example have a look at the tests.

Anyway, I'm with you that the projects will definitely benefit from each other.

What do you think about the separation into multiple packages? Are there any drawbacks?

christian-bromann commented 8 years ago

Sorry for getting back to you so late.

What do you think about the separation into multiple packages?

Yeah this is the plan. Ultimately it would be great to see similar modular structure like in WebdriverIO where you can apply reporters and service as you want. For example let's have data store addons to save images on the file system, on dropbox or any other arbitrary server. A different type of addon could be the comparison mechanism. Options are pixel by pixel, gm and applitools. Each one of these plugins could live in its own repository.

Here is a great graphic @TheSavior did once: http://slides.com/eliwhite/visualtesting#/14

Question is: do we want to have another organization for webdrivercss or is it ok to put it there? Also is wdio-screenshot only the screenshot taker and webdrivercss should manage this whole plugin system? What do you think about this plugin system?

elicwhite commented 8 years ago

Wow, cool to see people are still talking about this.

I actually built the project that follows the flow chart that @christian-bromann linked to. All the pieces live here: https://github.com/thea-diffing

It would be great to figure out how to work together to build the platform we all want to see.

zinserjan commented 8 years ago

Yeah this is the plan. Ultimately it would be great to see similar modular structure like in WebdriverIO where you can apply reporters and service as you want. For example let's have data store addons to save images on the file system, on dropbox or any other arbitrary server. A different type of addon could be the comparison mechanism. Options are pixel by pixel, gm and applitools. Each one of these plugins could live in its own repository.

That's what I already tried with wdio-visual-regression-service. It's responsible for taking screenshots with wdio-screenshot in different resolutions and provides a mechanism which I called compare methods to control the storage and diffing of screenshots.

Such a method is just a simple implementation of 4 different hooks:

At the moment I just implemented a LocalCompare method which just stores screenshots locally and compares them against some reference shots.

Question is: do we want to have another organization for webdrivercss or is it ok to put it there?

What do mean with "put it there"? WebdriverIO?

Also is wdio-screenshot only the screenshot taker and webdrivercss should manage this whole plugin system?

That was my plan. My personal opinion is that the screenshot taking functionality is so complex that it should exist on it's own. This makes it also easier to use it for other projects, too.

christian-bromann commented 8 years ago

That's what I already tried with wdio-visual-regression-service.

This service only works if you run it with the wdio testrunner. Something that would be great is that we provide VRT functionality for standalone as well as for testrunner mode. This would allow 3rd party libraries like Spectron or Concept to leverage from it. I would almost say that the service could be the next version of WebdriverCSS if we change it to work as a plugin not as a service. However I don't want to do that if you don't want that. I just think that more people know WebdriverCSS as that VRT tool for WebdriverIO.

What do mean with "put it there"? WebdriverIO?

I believe we get more user traktion if we have a GitHub organization for all plugins and tools around VRT with WebdriverIO. Maybe we can just keep it under the WebdriverIO org hat. But like I said, if you would rather want continue this under your account that is fine too.

the screenshot taking functionality is so complex

I absolutely agree. One of the reason why I stand off of it. It took too much time for me to fix all the issues especially on mobile.

zinserjan commented 8 years ago

Something that would be great is that we provide VRT functionality for standalone as well as for testrunner mode. ... I would almost say that the service could be the next version of WebdriverCSS if we change it to work as a plugin not as a service.

Sounds good to me.

I believe we get more user traktion if we have a GitHub organization for all plugins and tools around VRT with WebdriverIO.

Makes sense. I'm definitely open for this.

Question is: Does it make sense to create another organization for this? Another organization just for regression testing gives us probably a better overview. But this dependents on the planned features/plugins. I think we can keep everything as it is until the roadmap is clear.

As next step we should start collecting some ideas. Don't know what's the best way to do this.