search

PostHog / posthog

🦔 PostHog provides open-source product analytics, session recording, feature flagging and A/B testing that you can self-host.
https://posthog.com
Other
21.25k stars 1.26k forks source link

Allow rollout of different values to different groups in a "Multivariate Test" #16004

Open CodySchaaf opened 1 year ago

CodySchaaf commented 1 year ago

Is your feature request related to a problem?

We cannot currently customize features for different groups using multivariate testing.

Describe the solution you'd like

I'd like to specify the percentage of each multivariate test specific to each group.

Describe alternatives you've considered

N/A

Additional context

We currently use this feature to customize different "video based policies" (using aws chime for video conferencing) for different partners (groups) depending on their needs.

Thank you for your feature request – we love each and every one!

neilkakkar commented 1 year ago

Hey @CodySchaaf could you elaborate what this means?

I'm not clear what you want, but would variant overrides solve this for you: i.e. given a group, you can explicitly set them to belong to a variant, separate from the general rollout distribution. See: https://posthog.com/docs/experiments/common-questions#how-do-i-assign-a-specific-person-to-controltest-variant-in-an-experiment