Sprint December 11 - December 22

[mariusandra]

Global Sprint Planning

3 things that might take us down

1. 2. 3.

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

# Team ___

**Support hero:** ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red

### High priority

-

### Low priority / side quests

-

[mariusandra]

Project Query Engine

Support hero for Product Analytics: Michael 1st week, Robbie 2nd week.

Retro

• 🟢 HogQL BI features @Gilbert09
  • Great progress! You can now make line graphs for any HogQL query.
  • Still cleaning up this week, plan to launch before Tom goes on holiday.
• 🟡 Convert retention insight @webjunkie and launch trends/lifecycle @Gilbert09 @mariusandra
  • Lifecycle released to our team (today, finally)
  • Retention almost done
  • Trends on standby, waiting for lifecycle to be out, needs more testing.

Hang over items from previous sprint

Insights and BI.

OKR

Project "Finish the darned Query Engine" (Tom, Julian, Marius)

• 🟣 Finish insight HogQL conversion.
  • We're on track this year to convert everything except funnels
  • ✅ Lifecycle - released to our team
  • 🟢 Trends - 95% (to test more and release, done this week?)
  • 🟢 Retention - 90% (missing persons modal, group support, done this week?)
  • 🟡 Stickiness - 0% (not started, likely to finish this year)
  • 🟡 Paths - 0% (not started, likely to finish this year)
  • 🔴 Funnels - 0% (not started, unlikely to finish this year)
• 🟢 Support non-event data sources on HogQL insights (BI tools for data warehouse).
  • Great progress last sprint.
• 🔴 Build tools to measure HogQL query performance, and catch regressions before they hit users.
  • We'll fail at this... again. We keep addressing performance issues as users report, but didn't get to building proactive tooling.

Next sprint

• @Gilbert09 will only work 2 days this sprint
• @webjunkie and @mariusandra will be here until Christmas

High priority

To be revised/discussed during standup, but here are some ideas:

• Release BI to some users @Gilbert09
• Get retention and trends released. @mariusandra
• Great docs for the /api/:project_id/query endpoint, including HogQL and other data exploration nodes. @webjunkie

Low priority / side quests

• Performance work as it comes in.
• Work on the last 3 insights. I'd start with 1) stickiness, 2) paths, and leave 3) funnels to next year.
• Rolling out async queries and, related to that, updating Celery to have more queues.
• Django upgrades

[robbie-c]

Team Web Analytics

Support hero: @robbie-c

Hang over items from previous sprint

• Simplified traffic source

OKR

🟢 Build the dashboards to match 80% of functionality - close but a few big things remain ⚪ Start working towards the architecture to scale to a general release - deprioritised

High priority

• Continue working through https://github.com/PostHog/posthog/issues/18547
• e.g. finish the simplified traffic source
• sites scene https://github.com/PostHog/posthog/issues/12181

[benjackwhite]

Team Replay

Support hero: @daibhin

Retro

• N/A

Hang over items from previous sprint

• 🟢 Collect and playback wireframe recordings in PostHog player @marandaneto @pauldambra

OKR

• 📱 Mobile Replay 🟡
  • Android PoC (alpha) is the goal for end of year

High priority

• Q1 Planning
• Alpha release for Android replay that we can start trialling with people for feedback @marandaneto @pauldambra

Low priority / side quests

• Get a few customers onto network payloads @pauldambra

[EDsCODE]

Team Data Warehouse

OKR Q4

Objective

PostHog users can effortlessly bring their business data to PostHog and analyze it 🟢

• Key Results:
  • Be able to import data into data warehouse in one click from most popular SaaS tools (X, Y, Z connectors built)
  • 5 happy users/teams who have imported external data
  • Data Warehouse queries are compatible with insights hogQL

PostHog is moved over to whatever CH infrastructure we will be using for the next 1 year (at least) 🟡

• Key Results:
  • We are either on new infrastructure for EU that we host or on ClickHouse Cloud
  • Queries for large customers are performing faster
  • We can scale more easily (without James bespoke effort)

Retro

Product

• [x] Deploy either airbyte system or inhouse system (RFC here

Infra

• [x] Continue performance improvements with CH.cloud's help
• [x] More investigation and experiments with Projections
• [x] Complete benchmark of d3en.12xlarge
• [ ] Begin TF + Ansible configs for EU on prem cluster

High priority

Product

• [ ] Onboard 5-10 beta users
• [ ] track row counts
• [ ] Better chunking solution
• [ ] Custom field selection in UI (choose what sources to return from connector)

Side quests

• [ ] Start schema drift handling

Infra

• [ ] Complete EC2 Instance type benchmarking
• [ ] Begin TF + Ansible configs for EU on prem cluster
• [ ] Add projection for historical exports

[ellie]

Team Infra

Support hero: @frankh

High priority

• wrapping up capture, tracing, and anything else I've almost finished @ellie
• experimenting with new ingress @frankh
• new clickhouse setup @danielxnj

[Twixes]

Team Noteforce 3000

Support hero: all of us this time around

Retro

• Roll out internally and follow up on any feedback / issues ✅
• Get docs and screenshots up to date ❌
• Roll experiment for everybody (end of sprint) ✅

High priority

• Monitoring the impact of 3000 UI on usage via the experiment
• Resolving all non-blocking to-dos from https://github.com/PostHog/posthog/issues/18089 (Base 3000 section)
  • Docs and screenshots are carried over as part of this

Low priority / side quests

• Exploring further-out 3000 ideas, and the future of Noteforce 3000

[liyiy]

Team Feature Success

Support hero: @liyiy + @neilkakkar Days off: Juraj: 0.5 days (making a trek to the London christmas dinner!) Neil: 7 days Li: 4 days in the second half of the sprint

Retro

• Survey permanent feedback widget - @liyiy 🟢 will be done by the end of the sprint
• Survey quota limiting - @liyiy ⚪ did not get to this as it doesn't seem to be much of a problem lately and there's already billing quota limit
• Scheduled flag changes (and potentially everything else): 🟡 https://github.com/PostHog/posthog/issues/13842 - @jurajmajerik (on support) UI ready, moving on to backend.

Hang over items from previous sprint

• Make progress on SDK updates for relative date targeting & targeting by distinct id - @neilkakkar

---

OKRs

1. Get 5 reference customers for surveys
2. Get reference customers for feature flags and experiments

High priority

• Scheduled flag changes (and potentially everything else): https://github.com/PostHog/posthog/issues/13842 - @jurajmajerik
• Surveys easy cohort creation from survey events - @liyiy

Low priority / side quests

[davemurphysf]

Team Pipeline

Support hero: @tiina303 then @tkaemming

Retro

High priority

• 🟡 Capture-rs @xvello
  • 🟡 Fixed several causes of data drop and rolled out to 25% of posthog-js traffic (5% of all events), error rate slow and stable. Plan is 100% except big teams tomorrow, big teams next week.
  • ⚠️ More progress will require investment in our ingress for a safer transition (see https://github.com/PostHog/product-internal/issues/541#issuecomment-1843092527)
  • ✅ Instrument all the special-cases and kludges in capture-py to check whether they need porting
• 🟡 new pipeline UI ready to roll out to users @tiina303
  • ✅ metrics and logs tabs for apps
  • ✅ apps management page
  • 🟡 destinations list view WIP
  • 🔴 haven't started configs updating tab
• 🟡 elements_chain 100% rollout to all new posthog-js version @davemurphysf
  • Turned on Netdata test/staging accounts so they can test new behavior (https://github.com/netdata/posthog-netdata-event-processing/issues/136)
• ✅ Book venue/lodging for Jan 22-26 offsite @davemurphysf
• 🟡 PoE deferred write path to person_overrides for our team @tkaemming
  • Wrapping up PR to add worker to plugin server, close to on target but still running a bit behind
• ✅ Webhooks delivery system: scaffolding @ellie & database schema @tomasfarias
  • Scaffolding: done
  • DB schema: done
  • Consumer PR opened today

Low priority / side quests

• 🙇 Kudos to @MarconLP for the work with support, it was hard keeping up without you!
• 🔴 capture-rs: end2end probe revamp @xvello
  • not started, still important to allow sunsetting graphile and improve trust in the metric
• ✅ Make progress on CDP pricing RFC @davemurphysf
  • This was asked to be accelerated; Pricing RFC is up here (https://github.com/PostHog/product-internal/pull/533)
• ✅ Kill exportEvents & createBuffer @tiina303
• ✅ Removed StatsD pipeline metrics; new dashes in Victoria -> Ingestion @bretthoerner (not @brett)

Hang over items from previous sprint

• capture-rs: end2end probe revamp @xvello
• pipeline-ui: but this will shift to Q1 as @tiina303 will be on vacation
• element_chain: get rollout to 100%, by 12/15, exclusions done by EOY @davemurphysf

OKR

OKR, status (red/yellow/green) and action points if yellow/red

• 🟢 keep the lights on @team-pipeline
  • We’re still ahead of the scaling curve but volumes are increasing again
  • Some risks (unbounded datasets, domino incident from cohorts & groups) are still un-addressed and need to be re-evaluated for Q1
• ✅ Batch exports migrations complete 🎉 - @tomasfarias
• 🟡 Webhooks v2 @tiina303/@davemurphysf
  • Good progress on the new pipeline-ui side, but it's not quite ready to ship yet in 2023
  • delivery system: scaffolding and DB schema done
• 🟡 PoE @tkaemming
  • New path for team 2 will be in by EOW

High priority

1. Capture-rs @xvello
   • Finish rollout to big teams (make sure there’s no issues with proxies) end2end probe revamp
2. element_chain
   • rollout to 100% except exclusions @davemurphysf
3. POE
   • Simplify person_overrides schema to remove mapping table and drop exclusion constraint, migrate existing data to new table @tkaemming
   • Start running/testing squashes @tkaemming
4. Webhooks
   • Finish consumer PR (opened today)
   • Testing in general (load, etc…)
   • Provisioning

Low priority / side quests

[raquelmsmith]

Team Growth

Retro

Retro items

- Generally how we work: think about how our work aligns with objectives better - [ ] Launch data warehouse pricing https://github.com/PostHog/billing/issues/378 ...delayed by [airbyte](https://posthog.slack.com/archives/C019RAX2XBN/p1700603068979029) - [x] Alert people in UI when an initial billing limit has been sent @xrdt https://github.com/PostHog/posthog/pull/18668 - [x] Look into pinning first step in funnel @xrdt. The product analytics team is working a major insights refactor to make them HogQL backed so we are delaying any feature implementation. Neil gave a number of suggestions on how we might get around this with the current PostHog tooling. There are a number of pitfalls to his suggestions so we settled on a HogQL based solution for now. Tracking activation is really quite a non-trivial task. - [x] Reinstate recovery billing feature https://github.com/PostHog/billing/pull/409 @raquelmsmith - [x] Finish invoice split for upcoming invoices @raquelmsmith - [ ] Change MRR per product reporting to be calculated based on usage, not invoice line items @raquelmsmith Discussing [here](https://docs.google.com/document/d/1QY3BWjMfd6NvyeOSpPTDwB_zbCj5NPupBpf7Kke0UsY/edit) - [x] Onboarding config step for product analytics and session replay @raquelmsmith https://github.com/PostHog/posthog/pull/18720 - [x] Review 3000 onboarding and signup pages https://github.com/PostHog/posthog/pull/18746 @raquelmsmith - [x] org level info on users https://github.com/PostHog/posthog/pull/18332 @raquelmsmith - [x] @xrdt Figure out how to test unsubscribe survey events in cypress https://github.com/PostHog/posthog/pull/18805 - [ ] Think about how to better enforce API contracts between posthog FE and billing service @xrdt ?? Should we use graphql? - [x] Getting logged out when subscribing _locally only_ @xrdt. IDK why I'm being shown as anonymousUser but setting the `AUTO_LOGIN` env var resolves the surface level issue of having to manually log in every time I try to upgrade. - [x] It's not quite that simple to ensure that emails are [case-insensitive](https://stackoverflow.com/questions/50895643/removing-case-sensitivity-from-email-in-django-login-form/50895814#50895814)... - [x] ~Unset billing limit upon unsubscribe @xrdt~. Decided that this is probably a :sparks_joy: moment and can be left as is.

OKR

• 🟢 Objective 1: Nail real-time revenue reporting and forecasting
  • Reasoning: We rely on these metrics to know how we are doing.
  • [x] KR: Amortize invoices properly based on supply dates
  • [x] KR: Split invoices based on item supply dates
  • 🟡 Objective 2: Continue experimenting with onboarding and activation metrics
  • [x] KR: Product-specific onboarding performs better than the old onboarding flow
  • [ ] KR: 2 experiments for Product Analytics & Session Replay activation metrics each
  • [x] KR: Figure out how to track activation
  • Hogql queries
  • [ ] Make queries that "prove" our activation metric, show results for other combinations
    • Bianca has added column that shows retention for non-activated orgs
  • ❓ Bonus Objective 3: Better understand what leads to revenue
  • Reasoning: We're still a bit fuzzy about what metrics are leading indicators for revenue.
  • KR: A company scorecard with actionable insights that help us understand where to invest
  • KR: An understanding (doc or dashboard) of how people use multiple products
  • Charles is the primary on this, growth is just here to help

This sprint

High priority

• Activation
  • [x] Finalize activation work & RFC @xrdt
    • RFC is up.
  • [ ] Look into emails that happen after initial product analytics and session replay instrumentation, come up with an email to try to get people doing one of the activation events
    • run as experiment
    • @xrdt for product analytics
    • @raquelmsmith for session replay
• Other
  • MRR per product is wrong for feature flags
  • [x] Should we change MRR per product reporting to be calculated based on usage, not invoice line items? Discussing here. Shared doc with Neil.
  • [ ] Full testing of the onboarding flow @raquelmsmith
  • [ ] Make queries that "prove" our activation metric, show results for other combinations. There are some in this dashboard.
  • [ ] We shouldn’t show pricing above $20k/yr on the pricing pages so we don’t “shoot ourselves in the foot” for larger customers who will be paying above that
  • Could be an experiment, does it put people off from signing up because it isn’t as transparent, or feels enterprise-y.

Low priority / side quests

• [ ] Think about how to better enforce API contracts between posthog FE and billing service @xrdt ?? Should we use graphql?
  • [ ] Had discussed simply using errors from the serializer to make it more clear when there are mismatches in fields being requested from the front-end.
• [ ] Add metadata to startup plan customers in stripe
• [ ] ctas on billing for customers over 1k/mo
  • Bundling discounts
  • Team training
  • Roadmap chit chat

[brett]

@davemurphysf : @brett is actually a person, just not the one you want. Sprint looks great, though!

[fuziontech]

From now on can we get Brett's final approval on sprints?