Sprint - Sept 2 to Sept 13

[daibhin]

Global Sprint Planning

3 things that might take us down

1. 2. 3.

Urgent incident follow-ups

https://github.com/orgs/PostHog/projects/103/views/2

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

# Team ___

**Support hero:** ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red

### High priority

-

### Low priority / side quests

-

[daibhin]

Team Replay

Support hero: @pauldambra

• @annikaschmid - 1 week
• @neilkakkar - travelling 2 days

Retro

@marandaneto CS team helping a lot more with triage. Leaves more time to focus on technical investigations @daibhin error tracking has some momentum again. Neil working on Python & Ben helping with errors @neilkakkar nice to be back and productive

High priority

✅ done 🟢 in progress, on track 🟡 in progress, off track 🔴 missed

High priority

• ✅ Onboard @neilkakkar into the team
• 🔴 Network logs not capturing everything @pauldambra
  • Ticket investigated. Back & forth with customer but no resolution as of yet
• 🟡 React Native session recording @marandaneto
  • Progressing. Hoping it will be done. Will involve some copy & paste snippets
• 🔴 Masking and redacting Jetpack Compose views, issue @marandaneto
  • Focused on iOS blank screen issue

Error tracking

• 🔴 Alpha release @daibhin
  • error query is super slow. No limit. Need rewrite as a subquery
• 🟡 Alerting v1 @daibhin / @benjackwhite
  • Made a bunch of progress with CDP. Hoping to figure out PG write backs by end of the week
  • More complicated archiving / snoozing carried over to next week
• Build out SDK support
  • 🟢 Manual capture methods for JS and Python SDKs @daibhin
  • 🟢 Exception auto capture - Python - @neilkakkar
  • Django integration was something we hadn't anticipated
• 🔴 Demangling stack traces - @pauldambra
  • Speculative to begin with. Needs a bit more of a plan around it

Low priority / side quests

• 🔴 replay bug investigation tooling @pauldambra
  • Dropped in favour of more important
• ✅ convert session recording filtering to RecordingsQuery @daibhin
• 🟢 Flutter web using WebASM @marandaneto
• 🔴 Sample recordings, issue @daibhin
• 🔴 Hire iOS Engineer @marandaneto
  • pushed back to Q4
• ✅ Mobile session replay user interviews @marandaneto @annikaschmid
• 🟡 SwiftUI issues, [one] two @marandaneto
  • Investigated but don't see a way to resolve

OKR

1. OKR, status (red/yellow/green) and action points if yellow/red

• 📱Goal 1: People think of PostHog as a mobile solution 🟡
• 🪲 Goal 2: Error tracking in people's hands 🟢
• ⁉️ Goal 3: Hiring ✅
  • Neil's joining the team! 🟢
  • iOS Engineer 🔴

High priority

Mobile

• Masking and redacting Jetpack Compose views (issue) @marandaneto
• Support person_profiles in Android and iOS @marandaneto
• RN session replay more automatic via npm, not copying code snippets @marandaneto

Error tracking

• Django integration for autocapture @neilkakkar
• Rewrite events lookup as subquery @daibhin
• Per exception alerting configuration (snoozing / ignoring / archiving) @daibhin
• Filter exceptions by assignee @daibhin
• Stack traces @neilkakkar
  • Decide with CDP on approach for tackling stack traces
  • Begin implementation...

Session replay

• Network logs not capturing everything @pauldambra (?)
• Sessions >24 hours bug (browser online / offline events) @pauldambra (?)
• Dropped / compressed messages work revisited @pauldambra (?)

Low priority / side quests

• Review Replay UX issues & decide on quick wins @daibhin
• New replay filters (click counts, random sessions, new users) @daibhin
• Filtering internal users issue (PoE confusion)
• Integrate session recordings into error playlist (quick version) @daibhin

[benjackwhite]

Team Infrared

Retro / hangover

• 🔴 Add to django capture logging of payload shapes to determine long tail issues stopping us migrating to rust capture.
• 🟢 Migrate to rust capture for replay @frankh
• 🟢 Much better autoscaling for Mr Blobby - now it finally scales to match batch utilzation well
  • On for EU, will roll out for US
• 🟡 Migrate analytics to warpstream @danielxnj
  • Some pauses here as we wanted to be sure about issues around replay ingestion and some other configuration checks
  • Mirroring and ingestion is configured and ready
  • Delaying this to Q4 for actual rollout due to people availability
  • Clickhouse <> Warpstream stuff all seems fine

OKR

1. 🦹 Zero-trust security 🟢
2. 🤓 10x Developer Experience 🟡
3. 💪 Every service lives and dies alone 🟢
4. 💰 Save big on cost 🟡

High priority

• AWS costs exercise @frankh
• Write up a plan for how we will migrate Warpstream later @danielxnj
• Operational "handover" of working with infra world @frankh -> @Daesgar
• Comply with blockage of IPs from Russia @frankh
  • Add a region block notice to Django
  • Block if not logged in on all endpoints redirecting to page explaining why
  • If logged in also block but with team override (or something)
• PostHog JS - deploy its own assets @danielxnj
  • Move away from posthog being the source of truth for JS assets to posthog-js
• ArgoCD? @danielxnj
  • Get the installation up to date on management cluster connected to all envs
  • Get one service deploying to dev (and prod envs as far as possible)

[Phanatic]

Team Feature Success

Support hero: @Phanatic

Vacation

1. @dmarticus out to get married. 💍 Sept 3-7th
2. @annikaschmid : Sept 3rd to 11th.

Retro

• Customer support :
  • Overall really good.
  • Met a lot of SLAs
  • Automation to alert on SLA breaches is good.

✅ No-code experiments Toolbar : @Phanatic ✅ Feature flags rust rewrite, support person overrides: @dmarticus ⌛ Feature flags rust rewrite, support group properties overrides : @dmarticus ⌛ RFC for Supporting multiple experiment goals : @jurajmajerik ✅ Release the winning variant

Side Quests

✅ Updated Go SDK to support fallbacks to /decide endpoints if the Personal API key isn't provided, stood up CI and auto-versioning for this lib, and worked with some customers to merge in 3 new PRs to make the SDK better (good thing we got CI stood up before that!) @dmarticus ✅ fixed this person popover bug @dmarticus ✅ fixed the LemonInputSelector component that users were tweeting about @dmarticus

---

OKRs

1. Make sure feature flags can handle 10x current scale
2. No-code experiments ~3. Split out experiments into its own product~

High priority

• No-code experiments targeting & Support hero: @Phanatic
• Feature flags rust rewrite, early access, variant overrides, multivariate flags, evaluation reasons: @dmarticus
• Feature flags rust rewrite , point local posthog-js to the new rust endpoint : @Phanatic
• HogQL rewrite for experiments : @jurajmajerik

[annaszell]

Team ~Missing Michael~ Analytics of the Product

Support hero: Michael (& Anirudh) + Sandy (& Michael) Off: Thomas & Julian on parental leave, Anirudh out for 2 days

Retro

• 🟡 Alpha version of AI search-like querying for internal testing on Cloud – @skoob13 @Twixes
• 🟢 Reliability metrics – @anirudhpillai
• 🟢 Analytics alerts for internal PostHog dogfooding – @webjunkie
• 🔴 Simplify group analytics billing with @raquelmsmith – @Twixes
• 🍋‍🟩 Fix the funnel querying issues affecting high-value customers – @aspicer
• 🟢 Cache warming for all recently viewed shared insights - @webjunkie

OKR

1. Rock-solid analytics (@thmsobrmlr + @webjunkie + @aspicer + @anirudhpillai)
   1. 🟢 Legacy Minus – removing legacy insights code so that we can move fast
   2. 🟠 Tests Plus – shipping fewer bugs in the first place.
   3. 🍋‍🟩 Metrics Plus – catching issues before users report
   4. 🟡 Performance Plus - eliminating UX pain via maximum query performance/reliability, based on Metrics Plus data
   5. 🟢 Support Plus – sparking joy for users when they’re led to report a bug
2. Answering more product questions, deeper (@thmsobrmlr + @webjunkie + @aspicer + @anirudhpillai)
   1. 🔴 Growth Plus - increasing ease of onboarding, and subsequent retention
   2. 🟡 Analysis Plus - answering more product questions, more deeply
3. 🟡 ArtificialHog (@Twixes + @skoob13) – an LLM-based chat-like interface for answering product questions.

High priority

• Alpha version of AI search-like querying for internal testing on Cloud @skoob13 @Twixes
• Analytics experience dashboard - @anirudhpillai
• Fix internal user filtering issue - @aspicer
• Gen AI competitor research & user interviews - @skoob13 @annaszell

Low priority / side quests

• Testing & deploying new funnels code - @aspicer
• feature requests around dashboard previewing filter changes - @anirudhpillai
• Upgrade to Django 5

[raquelmsmith]

Team Growth

Retro

Retro items

- @raquelmsmith - [x] Personless default RFC - [ ] Flag-specific proxy url - waiting on ben/infra for final solution - [ ] ~Start on https://github.com/PostHog/company-internal/issues/1452 ~ - [ ] ~Fix error that happens after creation, merge https://github.com/PostHog/posthog/pull/23069~ - [ ] Set up billing alerts - [x] 500s - [ ] What else do we want here? Celery queues? - [ ] ~Add LLM usage to usage reports for Lior~ - [x] https://github.com/PostHog/posthog-js/pull/1370 - [ ] Complete SAML tests - spent time but got nowhere This is probably too much for this sprint.. listed in roughly priority order - @zlwaterfield - [x] Finish up billing limit migration - [ ] ~Get the first version for subscription interval support out~ - [ ] ~Complete pre-work for teams annual plans around multiple stripe customers/subscriptions~ - [ ] ~Get teams annual plan in prod (behind flag)~ - [x] Make a plan for self-serve annual commitments (and start on some changes for it) - [x] Work w/ Mine to get ICP score flowing back into PostHog - [x] Work w/ Infra to get Celery using keda and new queues for rate limiting changes Other things on the go - [ ] migrate from fully free session replay plans - [ ] migration from mailgun to customer.io w/ Joe

Q3 Goals

✅=finished 🟡=in progress 🔴=won't finish ⚪=not started

1. 🟡 Make onboarding awesome for Product analytics and Data warehouse (Raquel)
2. 🟡 Support self-serve annual commitments (Zach)
3. 🟡 Dive into the data to understand our billing metrics and customers better (Zach)
4. ✅ Launch pricing for data warehouse (Raquel)
5. 🟡 Hire 2 people (one for billing, one for product focus)

This sprint

Time off:

• @zlwaterfield - 1/2 day

  • [x] Session replay migration from free plans
  • [x] Fix the off by one error for annual credits mrr splits
  • [ ] Self serve annual credits - get v1 done and behind a feature flag in productions (without the monthly distributions yet)
  • [ ] Update + design self serve annual credits monthly distributions
  • [ ] Confirm ICP scoring data from Mine and start deeper analysis
  • [ ] Email churned subscribers (on going)
  • [x] Don't auto turn of group analytics and data pipelines
  • [ ] Shadow 1-2 technicals, move Google doc for arch technical into https://github.com/PostHog/interview-test and add more context on answers. Improve some of the grammar in the challenge.

• @raquelmsmith

  • [ ] Fix invoice.mrr not being used for invoice_with_annual table
  • [x] Fix cypress test ffailures (phani actually figured it out)
  • [ ] Implement Personless default switch
  • [ ] Start on https://github.com/PostHog/company-internal/issues/1452
  • [ ] Take another stab at SAML tests
  • [ ] RFC for enterprise plan
  • [ ] Fix error that happens after creation, merge https://github.com/PostHog/posthog/pull/23069
  • [ ] Add LLM usage to usage reports for Lior
  • [ ] Move all activation metrics to backend tracking

[fuziontech]

Team Click Haus, Haus of the Hogs

OKR Q2 2024

Objective

James as a Service -> Clickhouse as a Service

• P0 tasks such as
  • 🟡 Deletes
  • 🟢 Keeping clusters happy
  • 🟢 Provisioning more disks
  • 🟢 Schema Reviews
  • 🟢 Debugging
  • 🟢 Performance < Thanks @tkaemming
  • 🟢 Backups/Restores
• Decide whether ByConity is the way forward
  • 🟢 Load it with data, set up
  • 🟡 Test performance, test the functionality/compatibility gaps
• IF ByConity works, migrate over to it
  • 🟡 Enumerate all functionality that doesn’t work and update the functions/contribute to ByConity
  • 🟡 Syntax
  • 🟡 If it works on metal, put it in k8s with Karpenter 🟢 Already on K8s
  • 🟡 Evaluate which nodes we should use
• IF ByConity doesn’t work, reshard US to look like EU cluster
  • 🟡 All clusters (Dev, US, EU) should be consistent in shape and topology. This will make it easier to manage and maintain the clusters and apply learnings from one cluster to another.
  • 🟡 We want all cluster operations to be automated and managed through some form of infra as code that is available in source control.
  • 🟡 Schema management on ClickHouse should be entirely automated and managed through source control with no exceptions. This includes Coordinator schemas.
  • 🟡 We should be able to spin up and down replicas of any cluster with no manual intervention.
  • 🟢 We should be able to upgrade ClickHouse versions with no manual intervention.
  • 🟡 We should have tooling / runbooks for resharding (if we continue down the current coordinator path)

Board

https://github.com/orgs/PostHog/projects/85/views/2

Retro

@Daesgar - Too much ByConity ;p On one side it's great, but the other I need @fuziontech's help. We need to start testing performance. Right now it's looking pretty bad, but just needs some investigation.

@fuziontech - Distribution queue lag was a real pain. We have a solution in place that looks good. Happy to have a path forward to protect us from building up any backlog here.

Board Snapshot

[robbie-c]

Team Wab Enelytics

Support hero: @robbie-c

Robbie out 6 days (from the 6th)

Retro

🟢 Conversion events tile 🟡 Page-level conversion goals 🔴 Save web analytics page state

🟢 Get US onto sessions v2 (EU has been on it for a while) 🟢 Test sessions v2 perf hack with team 2 (its's 40% faster) 🟢 Make posthog-js CI more robust 🟢 Improve posthog-js bot detection

OKR

1. Make querying fast enough for large customers
2. Heavily requested features
3. Work better with other products
4. Product and growth

High priority

• Page level conversion goals
• Delete v1 sessions table after US people have been on v2 for ~a week without incident.

Stretch goals

• Doc updates
  • Cross domain tracking
  • Changes to attribution
• Add event count map to sessions v2
• Start backfilling the 4 new columns in sessions v2 (will likely take a few weekends)

[benjackwhite]

Team Centennial Data Properties (CDP)

Support: @benjackwhite

Retro

• 🟡 Release Feature Preview and then general availability of Hog Functions @benjackwhite
• 🟡 Pricing plan for new functions so that we can actually migrate things over to it
  • Will stick with existing pricing - just need special template to migrate existing customers who were using free action webhooks
• 🟢 Implement lambdas/closures, add missing STL functions, and support all element property filters @mariusandra
• 🟢 Support Replay team with Exception alerts using HogFunctions @benjackwhite @mariusandra
• 🟡 Get duplicate service up and running on Cyloctron so we can test team by team
  • Most things are there and ready to go, just missing plugin-server side to actually hook into it
• 🟡 Property/Event definitions off the critical path @oliverb123
  • Tuning and testing of the running service to make sure it works as expected

OKR

(to be refactored)

High priority

• Metrics side of cyclotron @bretthoerner
  • Dead letter depth
  • Queue depth
  • Anything else we can think of
• Roll out rust Property/Event definitions service @oliverb123
• Get the new hog services running in a way that is ready to test cyclotron @benjackwhite
• Wrapping up remaining list of Destination UX work to get it rolled out @mariusandra

[EDsCODE]

Team Data <->

OKR Q2 2024

Objective

Query 3000

• Key Results:
  • Autocomplete
  • Increase general BI experience/product https://github.com/PostHog/meta/issues/157
  • Declutter the data warehouse UI and make the features intuitive to find

Data Modeling MVP

• Key Results:
  • Infrastructure decided and implemented
  • Integrating external data with feature flags
  • External data everywhere in insights/persons/cohorts
  • Get billing team to use modeling in posthog for their invoices_with_annual table

Retro

• [x] Data Modeling MVP/experimentation. Backend foundation is in place. Prioritize UI and materialization layer @tomasfarias @EDsCODE
• [ ] Batch exports support catch up @tomasfarias
• [x] Continued quality improvements with data warehousing @Gilbert09
  • more integrations
  • monitoring/improving compaction work
  • first class credential support

High Priority

• [ ] release data modeling beta
  • finish out UI @EDsCODE
  • first pass of materialization @tomasfarias
• Quality improvements
  • [ ] first class credential support
  • [ ] schema realignment when it's changed for a source (new schema or postgres table etc.)
  • [ ] high priority customer support (

Needs addressing

• can't guarantee clickhouse distributed table lag will always be stable so batch exports has a chance at missing events