PostHog / posthog

🦔 PostHog provides open-source web & product analytics, session recording, feature flagging and A/B testing that you can self-host. Get started - free.
https://posthog.com
Other
27.28k stars 1.74k forks source link

Revenue Analytics flag auto-disables #32271

Closed ajmnz closed 3 weeks ago

ajmnz commented 1 month ago

Bug Description

When the Revenue Analytics feature flag is enabled, it does not stay active. After enabling, it automatically reverts to the disabled state, preventing the feature from being used. This occurs consistently and without any user interaction after activation.

Revenue Analytics feature flag cannot be enabled. After toggling the switch, it automatically reverts to the disabled state, and no error message is shown. This occurs consistently in Firefox, Chrome and Safari (macOS, latest versions).

How to reproduce

  1. Head over to the Revenue Analytics page
  2. Click on "Activate Revenue Analytics"
  3. Toggle the feature flag in the sidebar

Additional context

Attaching screen recording

https://github.com/user-attachments/assets/9bd9d4bc-7bc9-4585-9f8e-e9a5c157128f

Debug info

- [x] PostHog Cloud, Debug information:

Session: https://us.posthog.com/project/sTMFPsFhdP1Ssg/replay/0196d8c3-4093-7305-b13d-cbddf20543a7?t=620
Admin: http://go/adminOrgEU/0193bd33-f749-0000-ebaf-a5011f15ac55 (project ID 42932)
lshaowei18 commented 1 month ago

Additional context, not sure if it helps:

When the switch is toggled, a POST request is made. For the rest of the other flags that work correctly, the flag enabled value is updated correctly. One example:

"show-new-experimentation-engine-banner": {
        "key": "show-new-experimentation-engine-banner",
        "enabled": true,
        "variant": "enabled",
        "reason": {
            "code": "condition_match",
            "condition_index": 0,
            "description": "Matched condition set 1"
        },
        "metadata": {
            "id": 131714,
            "version": 5,
            "description": null,
            "payload": null
        }
    },

However, when revenue-analytics is toggled, the enabled will not be updated to true, it remains false.

lshaowei18 commented 1 month ago

I have one hypothesis, which is that currently for me I do not match the Release conditions set in the feature flag revenue-analytics-beta:

"revenue-analytics-beta": {
            "key": "revenue-analytics-beta",
            "enabled": false,
            "variant": null,
            "reason": {
                "code": "no_condition_match",
                "condition_index": 1,
                "description": "No matching condition set"
            },
            "metadata": {
                "id": 118067,
                "version": 21,
                "description": null,
                "payload": null
            }
        },

Specifically, I do not match the second condition that is set. I have tested this out on my own localhost, it seems like this reason is shown when a condition is not match.

Actionable for Posthog team

Could yall help to verify the release conditions for revenue-analytics-beta feature flag?

Juice10 commented 3 weeks ago

Same issue for me, when clicking the toggle often I got it to stick once, but when refreshing the page it was gone again.

Details:

lshaowei18 commented 3 weeks ago

Same issue for me, when clicking the toggle often I got it to stick once, but when refreshing the page it was gone again.

Details:

  • Posthog cloud
  • Mac OS Chrome/Safari

Hi @Juice10 , i believe a fix is incoming :) https://github.com/PostHog/posthog/pull/32825

rafaeelaudibert commented 3 weeks ago

Hey, team!

I lead the Revenue Analytics team. Yeah, #32825 should fix this. TLDR: We renamed the early access feature flag, and it turns out our code didn't handle that very well!

This should be merged by EOD. If you wanna get access to Revenue Analytics before than, just email me and I can get you in: [my first name] [at] posthog [dot] com

Juice10 commented 2 weeks ago

Thanks for fixing this, can confirmed it works now for me!