Automated cross-browser end-to-end testing (e2e) is like a gym membership. Everyone knows it's good for you. Just about everyone has tried it. But few manage to keep it up because it's hard and there is always something easier to do.
Unlike unit testing, which is fast, predictable, and full of good turnkey solutions, e2e testing is:
Flakey and full of false positives
Slow and cumbersome
Lacking in enterprise-grade solutions
Full of maddening quirks
...basically, it really really sucks to deal with. Most teams either give up, or say that they're doing it, when in reality their test suite is shallow and out-of-date.
To hell and back!
At WalmartLabs, we schlepped through the muddy trenches of e2e testing solutions for the past year, in search of a strategy that would be:
Collaborative: Allowing QA and Dev to break their silos and co-develop e2e test suites
Simple: Empowering folks with little programming experience to write powerful tests
Visible: Making e2e test results readily available, and easy to understand/debug
Stable: Without the usual false positives, glitches and quirks that erode confidence in e2e testing
Integrated: Part of the Continuous Integration workflow, alongside unit testing
We grew more gray hair, banged our heads on the desks, and cursed like sailors. But we wound up creating something called Magellan that we're damn proud of. It's less of a "framework" and more of an "ecosystem". It's a collection of tools, scripts, recommended best practices (and discouraged worst practices) that we've organically curated over time.
From inter-team to open-source
Since the beginning, we've treated Magellan like an "internal" open source project: striving to permit new WalmartLabs project teams to "jump right in" and adopt our e2e testing solution without a lot of hand-holding. We've gotten pretty good at it, and we'll soon be releasing the Magellan ecosystem as public FOSS. So we thought it would be a good time to share our stories.
What we'll show
We're not going to try to sell you a framework. But we want to tell the story of our journey to find all the missing pieces of the e2e testing puzzle--the key ingredients to a successful enterprise-grade e2e testing solution.
It's part technical, part politics, and all honest:
Explaining to developers why unit testing is not enough
Building a collaborative testing initiative between Dev & QA
Running a massively parallel suite, to compress many hours of test time into a few minutes
Leveraging lightning-fast PhantomJS builds to quickly catch egregious errors
Managing remote Browser-as-a-Service (BaaS) resources to do the heavy lifting
Justifying ROI to people who think this is all a waste of time
Speaker Bio
Dave is a Staff Software Engineer on the "Front End Ops" team at WalmartLabs. When he's not "automating the pain away" you can find him enjoying the Colorado wilderness with his family, and obsessing over espresso.
Public Speaking
I presented a Lightning Talk at Distill 2013. You can see it here (skip to 45:38)
Why everyone gives up on end-to-end testing (and why you don't have to!)
The story you'd like to tell
A tale of ten browsers
Automated cross-browser end-to-end testing (e2e) is like a gym membership. Everyone knows it's good for you. Just about everyone has tried it. But few manage to keep it up because it's hard and there is always something easier to do.
Unlike unit testing, which is fast, predictable, and full of good turnkey solutions, e2e testing is:
...basically, it really really sucks to deal with. Most teams either give up, or say that they're doing it, when in reality their test suite is shallow and out-of-date.
To hell and back!
At WalmartLabs, we schlepped through the muddy trenches of e2e testing solutions for the past year, in search of a strategy that would be:
We grew more gray hair, banged our heads on the desks, and cursed like sailors. But we wound up creating something called Magellan that we're damn proud of. It's less of a "framework" and more of an "ecosystem". It's a collection of tools, scripts, recommended best practices (and discouraged worst practices) that we've organically curated over time.
From inter-team to open-source
Since the beginning, we've treated Magellan like an "internal" open source project: striving to permit new WalmartLabs project teams to "jump right in" and adopt our e2e testing solution without a lot of hand-holding. We've gotten pretty good at it, and we'll soon be releasing the Magellan ecosystem as public FOSS. So we thought it would be a good time to share our stories.
What we'll show
We're not going to try to sell you a framework. But we want to tell the story of our journey to find all the missing pieces of the e2e testing puzzle--the key ingredients to a successful enterprise-grade e2e testing solution.
It's part technical, part politics, and all honest:
Speaker Bio
Dave is a Staff Software Engineer on the "Front End Ops" team at WalmartLabs. When he's not "automating the pain away" you can find him enjoying the Colorado wilderness with his family, and obsessing over espresso.
Public Speaking