search

ProvidenceGeeks / website-docs

Top level README and documentation space for development of the Providence Geeks website
https://www.pvdgeeks.org
4 stars 2 forks source link

Create a browser / device testing matrix #37

Open thescientist13 opened 6 years ago

thescientist13 commented 6 years ago

Type of Change

Summary

  1. Include in the Testing Guide a matrix of supported / required browsers and devices that all features should be tested against
thescientist13 commented 6 years ago

@saimatsumoto removing sprint3-carryover since it wasn't initially scoped to be part of Sprint 3

saimatsumoto commented 6 years ago

@thescientist13 @ekrasner1 Please review below device testing matrix and provide any feedback.

Test device matrix

Device, OS / Browser Safari 10 Chrome 62 FireFox 57 IE 11
OS X 10.11+ 7 (Medium) 10 (High) 3 (Low) N/A
Windows 10 N/A 10 (High) 3 (Low) 3 (Low)
iPhone iOS 10 7 (Medium) 3 (Low) N/A N/A
iPhone iOS 11 10 (High) 7 (Medium) N/A N/A
Galaxy mobile Android 6+ N/A 10 (High) N/A N/A
iPad iOS 11 3 (Low) 3 (Low) N/A N/A
Tablet Android 5+ N/A 7 (Medium) N/A N/A
thescientist13 commented 6 years ago

@saimatsumoto This is a great start! My one comment would be how to interpret the 1 - 10 scale? Maybe you could expand on it for us? What is a 3 or 7 or 10 telling me about what set of behaviors / features / etc that browser should support?

From my experience, what I look to a Testing Matrix is for these answers to a question about a defect / story. So if imagine scenarios like whether the Header or Events List is broken...

  1. Do we even support that browser / device?
  2. If so, should the feature be expected to work in that browser / device?
  3. If it should work in that browser, how big of a deal is it that it's broken? A P0 show stopping production hotfix, or something that can wait till the next release.

So ultimately, we're trying to build up a mapping of features (like we would capture in our E2E tests) and how bad it is if it's broken in production, so the scale should roughly allow someone to understand something like:

this device / browser should work for 80% of all user stories (starting at the top with with the most important)

saimatsumoto commented 6 years ago

Thank you @thescientist13 will make changes after getting more insight for this later today.