search

uber / react-vis

Data Visualization Components
https://uber.github.io/react-vis
MIT License
8.73k stars 835 forks source link

RFC: Visual Regression Testing #224

Closed amilajack closed 6 years ago

amilajack commented 7 years ago

A pretty powerful way of testing with confidence is visual regression testing. What are thoughts on this? uber/image-diff can be used to diff screenshots of images.

mcnuttandrew commented 7 years ago

I think you are right that visual regression testing can be a really powerful way to guarantee repeatable graphics. That said, I think we should work on making the rest of the code better tested before we invest in something heavy like screenshot diffing?

mcnuttandrew commented 7 years ago

Worth noting: image-diff seems like it is either deprecated or about to be deprecated.

mcnuttandrew commented 7 years ago

I've really enjoyed using phantom-css in the past, it works well and is pretty easy to reason about. I'm open to other suggestions if there are more modern alternatives?

amilajack commented 7 years ago

phantom-css seems to be pretty well maintained so i think its a good alternative

jckr commented 6 years ago

We're welcoming PR on this but it's not really blocking so will close for now.

ibgreen commented 6 years ago

Just FYI - The upcoming deck.gl 5.2 release includes a visual regression testing framework for WebGL layers, using puppeteer and chrome dev tools. Pretty sure many parts could be reused if there was interest.

jckr commented 6 years ago

maybe we can spin it into a micro framework

ibgreen commented 6 years ago

The basic BrowserDrivers parts are available in probe.gl.

Current plan is to expose image diffing tools as part of a collection of GPGPU tools in luma.gl, but we could discuss other options.

The test scene renderer is harder to share as it would need to be adapted for react-vis, but deck.gl's version could be used as a base.

ericsoco commented 6 years ago

I know @btford was exploring a similar testing strategy for another application a couple months back, we should sync our efforts.

On Fri, Mar 9, 2018 at 4:31 PM Ib Green notifications@github.com wrote:

The basic BrowserDrivers parts are available in probe.gl https://uber-web.github.io/probe.gl/#/documentation/api-reference-testing/browserdriver .

Current plan is to expose image diffing tools as part of a collection of GPGPU tools in luma.gl, but we could discuss other options.

The test scene renderer is harder to share as it would need to be adapted for react-vis, but deck.gl's version could be used as a base.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/uber/react-vis/issues/224#issuecomment-371983840, or mute the thread https://github.com/notifications/unsubscribe-auth/ABEzW3Zbgn_bkPaZjfE_VbxxwIjnX4OQks5tcx7ggaJpZM4LgNxJ .

ibgreen commented 6 years ago

Sounds good. Just in case, I added the image diffing tools as experimental exports in probe.gl.

I think a skeleton of the code that asynchronously drives multiple react renders and wait for comparisons could also be made reusable. Doing that could save a lot of time for others, but preparing it would take a little more time than I have right now, so let's sync first.