search

dwyl / technology-stack

🚀 Detailed description + diagram of the Open Source Technology Stack we use for dwyl projects.
288 stars 26 forks source link

Research: Browser and End to end testing #52

Open samhstn opened 7 years ago

samhstn commented 7 years ago

In general what will we be looking to use for browser and end to end testing in our projects?

Nightwatchjs, Nightmarejs with Daydream, Karma?

The above have been tried out in a few projects, but what do we feel will be our go to framework in the future?

nelsonic commented 7 years ago

@shouston3 great question! I've been leaning toward Nightmare because it's (a lot) faster than Nightwatch (which uses WebDriver) or Karma (which uses PhantomJS), and AFAIK Nightmare syntax looks more intuitive than Karma (which I admittedly only used back when I was doing Angular but looking at the Docs https://karma-runner.github.io/1.0/index.html they haven't changed...)

If you are investing some time to investigate this today, @tonydaly suggested looking at hound and wallaby in https://github.com/healthlocker/healthlocker/issues/201#issuecomment-284732335 both are worth a look because they are written in elixir which means we can write one language for our app & end-to-end tests.

My main concern is the test execution speed. ask @naazy, @Conorc1000 & @roryc89 about Nightmare (i.e. WebDriver) based execution speed. On their (admittedly "mature") project, the tests run reasonably fast on localhost but as soon as we attempt to run them on Saucelabs they go pretty slowly ... waiting 20mins for a PR to run is pretty painful. 😢

Could you please investigate if we can run Nightmare tests on Saucelabs? or if there is an alternative "cloud" service we can use to test on several browsers (Internet Explorer is the bane of our existence! because even though MSFT no longer support it, many Public Sector users still use IE8 in 2017!!)

FYI: @iteles for OxAbs, we need to sign up to a paid plan for SauceLabs ... we probably need to pony up $149/month: https://saucelabs.com/pricing

nelsonic commented 6 years ago

This issue has been open for a while and I feel it deserves an update for anyone else reading ...

image https://twitter.com/samccone/status/728412397426253825

We have been using Cypress.io for front-end testing on several Client and Open Source projects for the past few months and while it is far from "perfect" the overall experience/feedback has been positive. 👍 https://github.com/dwyl/learn-cypress.io

One of our Client Projects (coincidentally the one @samhstn (OP) is working on, though the test headache is not his fault) has had "issues" with Cypress ... specifically "intermittent test failure" due to a "nested" app Someone on sed project suggested using http://webdriver.io instead of Cypress.io https://github.com/webdriverio/webdriverio it sounds/looks good on the surface ... But a "red flag" for me is the fact that a testing tool has a build error on master... 😕 image

I have not been actively looking for an alternative to Cypress.io because Cypress is fast and the output (test video) is "product owner friendly". But as always, I have Open Mind if anyone has any suggestions for something better.

nelsonic commented 2 years ago

Considering Wallaby https://github.com/dwyl/learn-phoenix-framework/issues/149