Sprint 1.42.0 2/2 - Nov 14 - Nov 25

[timgl]

Global Sprint Planning

3 things that might take us down

1. Everyone has a different idea on what are the company goals.
2. Postgres person table data growth @fuziontech

Retro: Status of Outcomes from Previous Sprint

1.Saving groups of recordings for easy future finding / playback @alexkim205 - Nearly done. Just some premium feature limiting to do. Out by end of sprint

2. Supporting Billing rollout @benjackwhite - Supported 👍
3. Expose more data in the persons modal (ie how many times the event was triggered) + sql refactor @Twixes > backend is ready. Will be deployed Monday.
4. Finish data exploration view and put live, also migrate existing insight views. @mariusandra > Not live. A lot of work on the RFC and alignment.
5. Robust app failure handling @tiina303 > reprioritized. Worked on groups on events and person deletion data integrity issue.
6. Removing any dependencies on Postgres availability from the pipeline @yakkomajuri > Done for plugin server. PR for capture endpoint this week.
7. OpenSSL security fix (whole team effort) @guidoiaquinti > done
8. Explore AWS cost reduction @ellie > reduced by $10k, plan to reduce a bit further
9. SOC2, 4-5 issues that we want to fix this sprint @danielxnj > all fixed
10. Permissioning on feature flags @liyiy > basic implementation should be good, roll over a little in the next sprint
11. UI updates to make 100% rollouts prominent + showing user blast radius @neilkakkar > didn't work on this at all. Got urgent feedback from customer for overrides.
12. Configure notifs on flag changes that people want to follow + alert when flag should be cleaned up @EDsCODE > didn't make sense until permissions. Released android updates instead.
13. Get clickhouse cloud ingestion working @fuziontech > No movement, a lot of fires and support hero. Put out RFC this week and putting another for language out today.
14. Add sessions recordings to onboarding flow @kappa90 > Did RFC, will get it out this sprint.

Retro: What can we do better next sprint?

1. Feels like a fair amount of incidents
   • Harry: historical imports have been on the rise

Support hero this sprint

Week 1: Michael Week 2: Neil

Plan: Proposed Goals for Next Sprint

Each goal should have a single owner. Owner can only be an engineer.

1. Refactor onboarding with other options than just ingestion @kappa90 why? current activation metrics are going down.
2. Sentry is clean and errors are actionable and monitored actively @tiina303 Why: So that we can actually be sensitive to new exceptions and thus spot and fix bugs more quickly
3. It is easy for anyone to understand the current status of the pipeline with respect to the target metrics (and easy for us to spot and debug issues) @yakkomajuri Why: For us and other teams to have a clear sense of success / failure, as well as be able to spot and fix issues quickly
4. "Static" Playlists - Youtube style, adding individual recordings to a list @benjackwhite Why? - Common workflow is finding N recordings that demonstrate some specific problem / idea. Also enables selective long-term storage
5. Youtube style link to specific point in the recording @benjackwhite Why? - This is possibly one of the most requested features from our users and should encourage sharing with teams
6. Data Exploration continued - events list breakdowns. @mariusandra Why? Make it easier to explore data.
7. Make insights query performance measurable @macobo Why? Long-term investment. Our metrics today on ClickHouse query performance show users experience should be great. This doesn't match reported user experience and there's a lot of FUD in terms of if/what problems exist wrt to insights performance. We want to have a product metric(s) around experience that help drill down what queries are slow for key clients to help know if and what we should be improving.
8. Test feature flag permissioning beta with users & grab feedback @liyiy why? often requested by users
9. : UI updates to make 100% rollouts prominent + detailed view of users who are hitting flags @neilkakkar why? reduce number of stale flags that stay open in a team's interface
10. Improve our metrics + alerting, specifically around Kafka and Clickhouse, based on action items from incidents @guidoiaquinti Why? we've had a few incidents recently that could have been caught earlier if the systems alerted us (eg, the zookeeper/clickhouse issue this morning)
11. Sort out access to current monitoring systems, as right now we have several all over the place @ellie Why? While we have our in-cluster monitoring, we also have metrics.posthog.net. This is awkwardly split. Our in cluster monitoring isn't accessible enough - we need to make sure the whole company is on the same page
12. More SOC2 work: @danielxnj why? to get us SOC2 certified so more people can use
13. Get new postgres instance replicating @ellie why? Move away from Heroku

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

Team ___

## Retro

<!-- Talk about what went well, what didn't go well and any actions to improve next time -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## Planning

<!-- Each item should have a single owner. Owner can only be an engineer. -->

### High priority

-

### Bugs

-

### Low priority / side quests

-

[timgl]

Team Growth

Retro

• Releasing billing which happened.

Hang over items from previous sprint

• Item 1. prioritised/deprioritise

Planning

High priority

• Tweaks to the current ingestion flow @raquelmsmith
• Option to create new project with new demo data @raquelmsmith
• Secondary onboarding activation tasks @kappa90
• Integrate customer.io @kappa90
• Billing migration to new system @kappa90

Bugs

-

Low priority / side quests

-

[yakkomajuri]

Team Pipeline

Goals

• Sentry is clean and errors are actionable and monitored actively @tiina303 Why: So that we can actually be sensitive to new exceptions and thus spot and fix bugs more quickly

• It is easy for anyone to understand the current status of the pipeline with respect to the target metrics (and easy for us to spot and debug issues) @yakkomajuri Why: For us and other teams to have a clear sense of success / failure, as well as be able to spot and fix issues quickly

If you're curious, we're targeting something like the image below, with the targets clearly specified, and links to other relevant graphs and docs

[benjackwhite]

Team PostHog TV

Retro

Hang over items from previous sprint

None

Planning

High priority

• "Static" Playlists - Youtube style, adding individual recordings to a list Why? - Common workflow is finding N recordings that demonstrate some specific problem / idea. Also enables selective long-term storage
• Youtube style link to specific point in the recording Why? - This is possibly one of the most requested features from our users and should encourage sharing with teams

Bugs

-

Low priority / side quests

• PoC - Recording (Comments) / Notes Why?
  • Comment on any point on any recording
  • "Discussions" tab to see all comments for all recordings

[mariusandra]

Team Product Analytics

Retro

• Expose more data in the persons modal (ie how many times the event was triggered) + sql refactor @Twixes
  • PR for backend, some tests failing. Frontend PR out. No significant refactor needed. Thus done.
• Finish data exploration view and put live, also migrate existing insight views. @mariusandra
  • Lots of progress on the RFC and alignment. Started to refactor existing insight filtering. Will carry on.

Planning

High priority

• Data Exploration continued - events list breakdowns. @mariusandra

  • Why? Make it easier to explore data.
  • The goal for this sprint is to spec out the query nodes, build a new query router (frontend and backend), use this to power the "live events" table, and add support for breakdowns and aggregations on the events list table. Stretch goal: pivot tables and formulas.

• Make insights query performance measurable @macobo

  • Why? Long-term investment. Our metrics today on ClickHouse query performance show users experience should be great. This doesn't match reported user experience and there's a lot of FUD in terms of if/what problems exist wrt to insights performance. We want to have a product metric(s) around experience that help drill down what queries are slow for key clients to help know if and what we should be improving.
  • Work might include a new RFC, benchmarking approach, caching improvements or query fixes as well.

Bugs

-

Low priority / side quests

-

[EDsCODE]

Team Experiments

Retro

• Working directly with Tarteel @neilkakkar
• Backend to permission system @liyiy
• Exposures on feature flags @EDsCODE
• Rolling out posthog-android (almost!) @EDsCODE

Hang over items from previous sprint

• The rest of permissioning work (interface)

Planning

High priority

• Goal: Test permissioning beta with users & grab feedback
• Goal: UI updates to make 100% rollouts prominent + detailed view of users who are hitting flags

Looking Ahead

• Prepare context and plan for cohort experience update
• Flutter client library feature flag support

[ellie]

🚀 Team Infrastructure 🚀

Retro

@danielxnj: Went well that we closed the infrastructure, code, and operation monitors related to SOC2. @guidoiaquinti: Progress being made, but still a lot of interrupt driven work. Need to invest more into infra monitoring, eg capacity planning as we don't know our current limits. Need more effective alerting. @ellie: Lots of interruptions this sprint, but happy to see the incident process working so now we have a good set of action items. Saved $10k, but also spent $6k on an instance for migrating out of heroku.

Hang over items from previous sprint

No hangovers 🙌

Planning

A couple of us are going to be in Barcelona for half next week, so will likely be pairing on some of our issues to get them sorted fast

• Improve our metrics + alerting, specifically around Kafka and Clickhouse, based on action items from incidents @guidoiaquinti Why?

  • we've had a few incidents recently that could have been caught earlier if the systems alerted us (eg, the zookeeper/clickhouse issue this morning)
  • we shouldn't find out about this being wrong from customers

• Sort out access to current monitoring systems, as right now we have several all over the place @ellie

  • Why? While we have our in-cluster monitoring, we also have metrics.posthog.net. This is awkwardly split. Our in cluster monitoring isn't accessible enough - we need to make sure the whole company is on the same page

• More SOC2 work: @danielxnj

  • Create DR procedure for critical databases
  • Close version control monitors
  • Review infra policies

• Get new postgres instance replicating @ellie

  • it is currently half-setup, and needs to be ingesting logs