PostHog / posthog.com

Official docs, website, and handbook for PostHog.
https://posthog.com
Other
404 stars 413 forks source link

Blog: What is user acceptance testing #7590

Open ivanagas opened 7 months ago

ivanagas commented 7 months ago

Summary

Write a short paragraph on what this article is about. If applicable, what's the opinion or point we want to make in this article?

What is user acceptance testing? How to do it successfully. What tools to use. Focus on the how to and tools, rather than examples. Need to figure out the right angle for this one specifically.

Maybe there is a followup for UAT template, example, and checklist using PostHog as a big example because these keywords have big traffic themselves.

Where will it be published?

select any that apply

  • [x] Blog
  • [ ] Founders Hub
  • [ ] Newsletter
  • [x] Product engineers Hub
  • [ ] Tutorials
  • [ ] Other (please specify)

Why type of article is this?

select any that apply

  • [ ] High intent (i.e. comparisons and similar)
  • [x] Brand / opinionated (how we work and why, etc.)
  • [x] High-level guide (concepts, frameworks, ideas, etc.)
  • [ ] Low-level guide (step-by-step guide / tutorial)
  • [ ] Other (please specify)

Who is the primary audience?

select any that apply

  • [ ] Founders
  • [x] Engineers
  • [ ] Growth
  • [ ] Marketing
  • [ ] HackerNews
  • [ ] Existing PostHog users
  • [x] Potential PostHog users

What (if any) keywords are we targeting?

list any that apply

user acceptance testing (22k/m), uat (165k/m)

Headline options

suggest a few angles

What is user acceptance testing (and how you can do it)

User acceptance testing successfully: How to and tools

How to do user acceptance testing (and the tools you need)

How we do user acceptance testing at PostHog (and how you can too)

Will it need custom art?

Outline (optional)

draft headings / questions you want to answer

  • What is user acceptance testing? Why do it?
  • Testing apps and websites to ensure they work for the intended users in the “real world”. Validate business case. Basically, user testing or end user testing. There is a lot of unnecessary jargon associated with it.
  • Relatively similar to [testing in production](https://posthog.com/product-engineers/testing-in-production)? How does it different from local, internal, beta testing?
  • User acceptance testing is much more defined. It has criteria it is looking for. Alpha/beta testing is exploratory. Beta testing can be a type of UAT.
  • Much more business-focused, user acceptance testing cares about how “business users” will react. Sometimes it is known as operation, contract, or regulation acceptance testing.
  • Figuring out what’s wrong and fixing it, includes UI/UX, performance.
  • The last phase of testing before release, after unit testing, Q/A, integration testing etc. Protects against releasing bad/broken software.
  • Corresponds to the requirements phase of the software development lifecycle. Validates that requirements have been met.
  • Comes from [(systems) engineering](https://en.wikipedia.org/wiki/Acceptance_testing)
  • ITSQB defines at:
  • “Formal testing with respect to user needs, requirements, and business processes conducted to determine whether a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether to accept the system.”
  • User acceptance testing should happen in real world scenarios. Should test user journeys. Matches user story.
  • What does a successful user acceptance test look like?
  • Have a plan for what requirements you need completed. For example, metrics + documentation + lack of show stopping bugs.
  • Give to users, recruit users, customers, do internal testing with people who don’t know the product. Maybe subject matter experts.
  • Check:
  • Application works as expected, completes the task users want it to complete.
  • Functionality is useful and aligns with business, strategy, goals.
  • Performance, bugs, visuals errors, defects, regressions fixed
  • Completed a checklist and ready for release.
  • Tools for user acceptance testing
  • You need to measure that your test is successful. PostHog has all the tools you need to do user acceptance testing. In a way, being all-in-one makes us the ultimate user acceptance testing tool.
  • Feature flags for rolling out features, beta management. Targeting the right target audience is important.
  • Session replay to see how users use it, do they understand the UI/UX, performance.
  • Survey to get qualitative feedback (book interviews, open feedback), report bugs.
  • Usage analytics, business analytics
  • How do we do user acceptance testing at PostHog? (maybe a different article)
  • We do user acceptance testing at PostHog, but we don’t really call it that nor do we have a formal process for it.
  • https://posthog.com/product-engineers/measuring-feature-success
  • Do we have happy users using the product? Does the team agree that it accomplishes the goal?
  • Are visual, performance bugs fixed?
  • We test in production and user PostHog.
andyvan-ph commented 7 months ago

I think we should pass on this for now:

  1. This feels more enterprise-y to me. Correct me if I'm wrong, but this is basically a corporate version of a beta test?
  2. There's lots of search, but I think it will be VERY hard to rank for this. It's very broad.
  3. Also feels more geared toward product managers rather than engineers. Would an engineer run a user acceptance test?