search

facebookarchive / huxley

A testing system for catching visual regressions in Web applications.
Apache License 2.0
4.07k stars 286 forks source link

Using Huxley on Pages with dynamic content #55

Open nikglavin opened 10 years ago

nikglavin commented 10 years ago

More of a question here, can't seem to find much info around or a forum for discussion so thought i'd ask it here.

Is there suggestions on how to setup huxley to run thorough CI on pages that are not static ? I have an issue where the test servers do not necessarily have the same database setup as the development environment, so there are regularly widgets that will not be in sync showing different data.

Would like to hear of ways people have been getting around this issue

plasticine commented 10 years ago

@nikglavin Extract common UI patterns out into isolated/reproducible examples, and test those. Testing whole pages is always going to have a higher rate of false positives and it too broad of an approach IMO.

nikglavin commented 10 years ago

I understand the idea of testing ui components, however do you test against "live" (component) templates, or are you just testing against a static template that has been mocked specifically for the test ?

This is how I had initially began to test but found that this is not ideal as it doesn't account for changes to the templates. Rather only test against changes to the css against the idilic structure of the component. Although still helpful this doesn't take into account changes to the structure of the "live" template.

For me to test against "live" components I would have to create some rather large clunky objects to provide a static set of data for each test.

I am interested to know how yourself or others get around or approach this problem ?

plasticine commented 10 years ago

My $0.02;

I understand the idea of testing ui components, however do you test against "live" (component) templates, or are you just testing against a static template that has been mocked specifically for the test ?

This is how I had initially began to test but found that this is not ideal as it doesn't account for changes to the templates. Rather only test against changes to the css against the idilic structure of the component. Although still helpful this doesn't take into account changes to the structure of the "live" template.

Yep, we use the exact same partials/presenters/templates as the ones in the app.

I work at a Ruby shop, and we’ve had a lot of success with tools like https://github.com/kneath/kss running within the app, and with rendering the component example using the exact Rails partial/presenter/whatever, exactly like we would in the app.

For me to test against "live" components I would have to create some rather large clunky objects to provide a static set of data for each test.

If you were writing unit tests you would always do this, testing the UI is no different really.

plasticine commented 10 years ago

I should also add though that we are not actually using Huxley for this at the moment, we’re using a homebaked setup that uses a bunch of the tools that are already in our testing toolchain; RSpec, Capybara, Selenium and Browserstack.