Cohort size different to the same user count in an insight

Bogdanovist commented 7 months ago

Bug Description

Bug description

I have defined an action corresponding to a sign up event of a specific user type. I have a cohort defined for any signed up users of this type. I've seen confusing results using this cohort. I also have a simple KPI insight that is the count of the unique users who have ever taken that action, and this number is around twice the size of the cohort. They should be the same?

How to reproduce

Links:

the action
the cohort this says 11754 users currently
the insight this says 23864, is should be the same value?

Debug info

Session: https://us.posthog.com/project/sTMFPsFhdP1Ssg/replay/018ef2b5-e529-789e-baab-dd1f2ec89d77?t=2149 (at https://us.posthog.com/project/20905/cohorts/67057#panel=support%3Asupport%3A%3A)
Admin: http://go/adminOrgUS/018652cc-68f9-0000-931c-c7875d4278cb (Project: 20905)
Sentry: http://go/sentryUS/20905

MarconLP commented 7 months ago

Hey @Bogdanovist, the timestamp property will look like this: 2024-04-18T20:47:17.945000Z. Therefore this filter won't work: timestamp < '2023-07-17'. Could you share where you got that syntax from?

Bogdanovist commented 7 months ago

hmm, I'm not sure. This was setup a long time ago. Looking at other Actions we've defined they use a similar approach. Woul there have been any changes to e.g. HogQL that would have broken this?

How should the syntax look to do this correctly?

Bogdanovist commented 7 months ago

This syntax seems to work in Insights, I just tried a random filter on a pageview chart and this worked timestamp > '2024-04-15'

mariusandra commented 7 months ago

Hey @Bogdanovist , I had a look.

I believe there's a bug with the cohort calculation somewhere. If I just look for account_created event over the last 30 days, then there are over 5000 users who have completed such an event. The cohort gives this amount if I use the event name directly. However when going through an action, insights show the number correctly, but cohorts for some reason remove half of the people.

We're still investigating as to why this might be. However for now, either use the event directly in the cohort, or use the action directly in the insight.

Bogdanovist commented 6 months ago

Thanks for the information, please keep me updated on any progress.

We used actions extensively in cohorts due to the limitations in cohort definitions in the past. It looks like this has been upgraded at some point and cohorts definitions have more nuanced filtering options. I can probably remove the reliance on Actions that we were forced to use in the past. But understanding what is going on here would be helpful.

Thanks

On Fri, 19 Apr 2024 at 18:14, Marius Andra @.***> wrote:

Hey @Bogdanovist https://github.com/Bogdanovist , I had a look.

I believe there's a bug with the cohort calculation somewhere. If I just look for account_created event over the last 30 days, then there are over 5000 users who have completed such an event. The cohort gives this amount if I use the event name directly. However when going through an action, insights show the number correctly, but cohorts for some reason remove half of the people.

We're still investigating as to why this might be. However for now, either use the event directly in the cohort, or use the action directly in the insight.

— Reply to this email directly, view it on GitHub https://github.com/PostHog/posthog/issues/21669#issuecomment-2066065536, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABEQKWJHP5URD4LBC6MRN5TY6DG6ZAVCNFSM6AAAAABGN6ZP2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRWGA3DKNJTGY . You are receiving this because you were mentioned.Message ID: @.***>

Bogdanovist commented 6 months ago

Just following up, there are still limitations in cohort filtering (such as not being able to filter "before" a given date, only after) that mean I cannot replicate the required Action logic to define the cohort I need without using an Action.

On Mon, 22 Apr 2024 at 09:53, Matt Francis @.***> wrote:

Thanks for the information, please keep me updated on any progress.

We used actions extensively in cohorts due to the limitations in cohort definitions in the past. It looks like this has been upgraded at some point and cohorts definitions have more nuanced filtering options. I can probably remove the reliance on Actions that we were forced to use in the past. But understanding what is going on here would be helpful.

Thanks

On Fri, 19 Apr 2024 at 18:14, Marius Andra @.***> wrote:

Hey @Bogdanovist https://github.com/Bogdanovist , I had a look.

I believe there's a bug with the cohort calculation somewhere. If I just look for account_created event over the last 30 days, then there are over 5000 users who have completed such an event. The cohort gives this amount if I use the event name directly. However when going through an action, insights show the number correctly, but cohorts for some reason remove half of the people.

We're still investigating as to why this might be. However for now, either use the event directly in the cohort, or use the action directly in the insight.

— Reply to this email directly, view it on GitHub https://github.com/PostHog/posthog/issues/21669#issuecomment-2066065536, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABEQKWJHP5URD4LBC6MRN5TY6DG6ZAVCNFSM6AAAAABGN6ZP2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRWGA3DKNJTGY . You are receiving this because you were mentioned.Message ID: @.***>

PostHog / posthog