garris / BackstopJS

Catch CSS curve balls.
http://backstopjs.org
MIT License
6.82k stars 607 forks source link

No support for CSS Flex-box model with phantomJs 2.1 #358

Closed gael-boyenval closed 7 years ago

gael-boyenval commented 8 years ago

As the title mention it, phantomJS does not support flex, and it is very very problematic… Making visual regression testing with a broken layout is not quite optimal.

The worst thing is that they even do not want to fix that before v2.5

Generaly speaking I think that PhantomJS is not the better choice for webkit based Headless Browser ( a lot of issues, no support for modern CSS).

For those who do not want to use a geko based headless browser (like me) I suggest you to look into electron (https://github.com/electron/electron) (large community, created and maintained by github, all modern browser features)

edit: not so sure about that…

regards

gael-boyenval commented 8 years ago

Edit : use nightmare => https://github.com/segmentio/nightmare#api simple API, under the hood it's electron. someone dev that, if that can be of any help : https://github.com/MisumiRize/nightmare-screenshot

carryonZhang commented 7 years ago

Hi! Do you konw how to put nightmare into backstop, I mean make nightmare to be a part of config of backstopjs.

carryonZhang commented 7 years ago

just install nightmare in local path, and change the property of "engine" to be nightmare?

gael-boyenval commented 7 years ago

@carryonZhang : no this will not work. If the Backstop team choose to implement nightmare, they will have to write compatibility code. That is not the case at the moment.

if your problem is relative to flex layout, I worked around it by setting the autoprefixer's browser-list with iOS 8.1 to get the -webkit-flex... prefixes.

"browserslist": [
      "> 1%",
      "last 2 versions",
      "iOS 8.1"
  ],

It doesn't feel right to have to generate more code than necessary in order to get backstop to display properly. :(

garris commented 7 years ago

Hi all. There are no plans currently for integrating electron or headless chrome. These are both very cool implantations/projects which I respect a lot -- unfortunately their APIs are not compatible with Casper.js and hence BackstopJS.

FWIW: With the apps I work on the flex prefixes are not a big deal, we include them as part of our build and the cost is negligible -- whereas being able to automatically check our entire app for visual regressions is a very big benefit -- so it is a pretty easy calculation for us.

Good luck with your project. Please let me know what you guys end up doing -- even if you decide to move to another solution -- it would be valuable feedback to me. Cheers!

gael-boyenval commented 7 years ago

@garris, thank's for the reply. I understand your position.

Yet I think that the headless browser you use for regression tests should not be the one you write compatibility CSS for. It should (ideally) being the reflect of the modern browser you target.

And Phantomjs has been very very slow at implementing new features, and seeing the number of non-solved issues, it seem that they will not catch up soon.

In the other hand electron is maintained by GitHub, very modern and known to be 2 times faster (in some cases).

I wrote a little test, using nightmarejs, nightmare-screenshot and blink-diff, and it seem promising. I'll show you the result if I go further.

carryonZhang commented 7 years ago

Thank you for your reply! It helps me to put my focus on another test, karma for Unit test with mocha. I thought I can complete the whole test system for my company though I haven't any actual experiences on automation testing before. I think the whole test system includes CSS test, JS test, and CI. For now, I'd better focus on how to complete JS test.

c-vetter commented 7 years ago

@gael-boyenval I can not confirm your assertion. Using phantomjs-prebuilt@2.1.14, the sample from http://codepen.io/team/css-tricks/pen/EKEYob?editors=1000 works. What am I missing?

gael-boyenval commented 7 years ago

@rasenplanscher : your sample is using bullet-proof vendor prefixes.

  display: -webkit-box;
  display: -moz-box;
  display: -ms-flexbox;
  display: -webkit-flex;
  display: flex;

  -webkit-flex-flow: row wrap;
c-vetter commented 7 years ago

And what is the issue with that? Your issue description was that phantomJS would not support flex-box, not that you would need prefixes.

gael-boyenval commented 7 years ago

@rasenplanscher flex-box is prefix free, like every CSS supported properties. Vendor prefixes are specific vendors implementations of flex-box, not flex-box. So yes phantomJS not supporting flex box is an absolutely valid issue description.

The problem, has I mentioned it in a previous message, is that I add auto prefixers to my build process. (like in any good css workflow). In order to make PhantomJS display flex correctly, I have to add old browser support to my autoprefixer, or manually add vendors prefixs (like in the Stone Age) . It's obviously more code or more work than required.

But I already said all that… ;-)

c-vetter commented 7 years ago

@gael-boyenval Sorry, I somehow didn't catch that 😬

c-vetter commented 7 years ago

I do second the point that the testing setup should ideally be as close to production as possible without sacrificing speed, which nightmare may accomplish very well.

garris commented 7 years ago

Just an update here. Chrome headless support has been added. Now all your designs should render without any extra effort for tests.

Please reopen if you still have any issue using chrome headless.

Enjoy.