Open codecovdesign opened 7 months ago
Understanding the Problem to Solve:
-- I'd additionally add the ability for microservices customers to have a meaningful view of "project coverage" as a repo is probably too small to represent a "project"
Who are the expected users of this page? What are their roles and needs? -- For orgs with large repos, it would be org leaders (such as directors and executives) -- For orgs withs microservices, it would also include eng managers or even individual users
Aligning with Business Goals:
How does this page contribute to our overall business objectives? -- We sell and differentiate well in monorepo customers with tools like flags and components -- We do not differentiate well against the OSS or paid competitors in orgs with small repos. Most other paid competitors also have a page like this so is considered "table stakes"
Understand coverage on a set of repos or more than one repo. Eventually it would be a goal to show flag and component coverage across multiple repos.
-- A key tool for "Pro" tier or greater.
-- It is demo'd in every sales pitch as a way to analyze coverage data. Therefore, customers expect it to work and to show overall weighted average project of the repo
How does this feature fit into the larger product direction and strategy?
Day 0: Don't show misleading data to users Long term: We need to differentiate in microservices customers more vs. only monorepos.
https://github.com/codecov/roadmap/issues/28 Breakdown of current functionality and problems @codecovdesign
Understanding the Problem to Solve:
Identifying the Target Users:
Aligning with Business Goals:
Defining User Goals:
<situation>
, I want to <motivation>
, so I can <expected outcome>
Clarifying How It's Being Sold:
Setting Customer Expectations:
<insert here>
Larger Direction and Strategy:
Feedback from Sentry team strategy, related to product equivalent and their direction in handling it:
Understanding the Problem to Solve:
Identifying the Target Users:
Aligning with Business Goals:
Defining User Goals:
<situation>
, I want to <motivation>
, so I can <expected outcome>
I'm responsible for a specific component across repos
, I want to views these flags and/or components
, so I can the testing across repos
I have code quality standards
, I want to be able to view data
, so I can see quality is improving
Clarifying How It's Being Sold:
Setting Customer Expectations:
Larger Direction and Strategy:
Understanding the Problem to Solve:
Identifying the Target Users:
Aligning with Business Goals:
Defining User Goals:
<situation>
, I want to <motivation>
, so I can <expected outcome>
evaluating a dependency
, I want to I want to see if idea is reliable
, so I can merge/use the library
I am conducting a post mortem on incident
, I want to understand what lines had caused incident
, so I can action appropriately
Clarifying How It's Being Sold:
Setting Customer Expectations:
Larger Direction and Strategy:
Understanding the Problem to Solve:
Identifying the Target Users:
Aligning with Business Goals:
Defining User Goals:
<situation>
, I want to <motivation>
, so I can <expected outcome>
Clarifying How It's Being Sold:
Setting Customer Expectations:
Larger Direction and Strategy:
The Analytics page in Codecov, intended for team leads and engineering managers overseeing multiple repositories, faces mixed perceptions regarding its utility and relevance. While some see it as crucial for reviewing code quality trends across repos, others question its value. The feature's ability to align with broader business objectives and cater to both developers and managerial personas remains a topic for strategic reevaluation; emphasizing the need for actionable insights and potential shifts towards a more developer-centric offerings. Based on the feedback/discussions there are follow up discovery issues listed below for consideration, if we’d like to explore this space further.
Problem Statement
The Analytics page at the organization level, has been facing functionality issues and user confusion since the initial iteration which had minimal changes since mvp and requires a thorough reassessment. Example the page, showing maximum data instead of average, may not be providing the intended insights, leading to confusion about its value. Furthermore, the page is not meeting customer expectations as promised in our sales.
While we have identified ideal short-term solutions (Roadmap Issue #28), a deeper understanding of the foundational principles behind this feature could be helpful to ensure long-term effectiveness and alignment with user/business needs/goals.
Proposed Solution: Rediscovering Foundations
Goal: to build consensus and reach alignment on Analytics as a team, so that we can understand how to best support and prioritize this feature moving forward. Let's investigation the foundational / first-principle aspects of the 'Analytics' section:
Understanding the Problem to Solve:
Identifying the Target Users:
Aligning with Business Goals:
Defining User Goals:
<situation>
, I want to<motivation>
, so I can expected outcome<expected outcome>
Clarifying How It's Being Sold:
Setting Customer Expectations:
Larger Direction and Strategy:
Action Items
<who else?>
Additional Notes