PostHog / posthog

🦔 PostHog provides open-source product analytics, session recording, feature flagging and A/B testing that you can self-host.
https://posthog.com
Other
21.6k stars 1.29k forks source link

Feature Flag Values - maybe? #3213

Closed IgnusG closed 2 years ago

IgnusG commented 3 years ago

Is your feature request related to a problem?

I was thinking about a way to test out a change of a value (let's say some weight input for an algorithm) only for specific users and feature flags seem perfect since they can be configured to split traffic as well as only apply to a sub-group of users.

Describe the solution you'd like

A way to set values for feature flags. There are a couple of questions I'm unsure about:

Describe alternatives you've considered

Multiple config endpoints coupled with a feature flag(s) - no granular changes possible since all adjustments require a copy of the whole config.

Maybe this is possible to implement as a plug-in? If so should it become part of core? If not is it worth implementing?

weyert commented 3 years ago

Yeah, this woud be useful to allow non-boolean values for feature flags.

For example, integer values or string values. I can imagine that supporting different variants for a feature flag would work, where you can define the true, and the false variant values of a feature flag, and specify which variant value should be consider the default.

I don't think this can be implemented via the PostHog plugin feature, though.

IgnusG commented 3 years ago

Hmm, since it cannot be a plugin this would have to be implemented in the core. I guess the next question would be which parts of the codebase (which files) need updating: I can imagine DB, the server UI & Logic, API, client libraries. I assume pretty much everything the feature flag feature touches

Another thing left to discuss is also how the UI should look like. It should probably be possible to specify which values to return to which users or define the split between these values.
The boolean now works (if I understand it correctly) - return true if user in specified group or in "positive" part of the split, otherwise false.

I assume the most flexible would be to therefore split it on different values: "string 1" -> user group 1, "string 2" -> user group 2, "default" -> otherwise or with splits: "string 1" -> 20%, "string 2" -> 40%, "default" -> 40%

weyert commented 3 years ago

Yeah, I think it would split up the feature flag between feature flag details and its variants so you would associate and variant to a feature flag. One of the variants could be marked as the fallback value, and the variant would have a value that could be boolean, sting or integer value, and then filters are associated with it.

I am wondering how other solutions are doing this

IgnusG commented 3 years ago

I found more providers and tried some of them. Here's a broad overview:

Optimizely
Requires an account. Lets you create feature flags with attached values - these can a string, boolean, integer, decimal, JSON. You can then specify groups of users where the feature should run (or everyone) and then you specify how much of those groups or how much of everyone it should run for. The values can be configured per group and for everyone. If the user falls in the negative sample they receive the default value.
ConfigCat
Has a demo app. Allows you to create features with a specific type (one of integer, boolean, string or decimal). You can then specify different rules how the traffic should be split and who should receive which values. It's very similar to Optimizely with the difference being that a feature flag has one type instead of a collection of them.
weyert commented 3 years ago

Cool, I think this would be feasible with the existing filter logic? I would imagine that the different variables for flag would require a restructuring of the identify-endpoint and the client libraries. Like the idea :D

Are you on the Posthog slack? Happy to chitchat more about it over there

IgnusG commented 3 years ago

Cool, I think this would be feasible with the existing filter logic? I would imagine that the different variables for flag would require a restructuring of the identify-endpoint and the client libraries. Like the idea :D

Are you on the Posthog slack? Happy to chitchat more about it over there

Happy to discuss it more on slack @weyert. I'll see if I can find you on there

jeduden commented 3 years ago

I am also looking into how get support variants. Currently, we have Firebase RemoteConfig (and which depends on google analytics). In order to fully remove Google Analytics, we would need to have all features of RemoteConfig replaced.

samwinslow commented 3 years ago

We recently released alpha support for multivariate flags - see: https://posthog.com/docs/user-guides/feature-flags#multivariate-feature-flags-alpha. You can access the string values on the client side via our JS SDK.

Currently this works like the percentage split example above ("string 1" -> 20%, "string 2" -> 40%, "string 3" -> 40%).

Please let me know if you'd like me to add you to the alpha testing group by emailing me at sam @ posthog.com!

paolodamico commented 2 years ago

Done and released to everyone!