openflagr / flagr

Flagr is a feature flagging, A/B testing and dynamic configuration microservice
https://openflagr.github.io/flagr
Apache License 2.0
2.43k stars 186 forks source link

Submit experiment results #185

Closed johnsardine closed 2 years ago

johnsardine commented 6 years ago

Flagr can decide which variation to use for an experiment but it does not receive the data generated by the experiment.

For example: If you're evaluating the time to complete a task given an A/B variant, you'd want to log how much time the variant A got vs the variant B.

Does flagr intend to collect these results in the near future? Is it open to contributions?

zhouzhuojie commented 6 years ago

That sounds like a good idea! And definitely open to contributions!

I think it's similar to track, so clients can send back to Flagr with additional evaluation context together with previous evaluation result.

I would propose

https://checkr.github.io/flagr/api_docs/#operation/postEvaluation

Let me know if you want to start working on it, or I will find some time later.

johnsardine commented 6 years ago

@zhouzhuojie Thank you for your prompt reply. I was thinking a bit more with my team and maybe you also reached this conclusion on your side. The actual collection of the results is probably not a priority for this project.

Imagine you collect metrics using Mixpanel, you can easily collect the experiment results there. Otherwise you'd have to develop a couple of analysis mechanisms into Flagr.

Thank you for looking into this but I'll actually drop this request unless you actually see the need for it.

zhouzhuojie commented 6 years ago

It's not a very high priority, and Flagr already collected server-side evaluation logs.

I would imagine this can be useful for the following scenarios:

I will keep this issue open in case someone or me wants to implement it.

github-actions[bot] commented 2 years ago

Stale issue message