"reasoning" concept for a flag evaluation

[justinabrahms]

When we evaluate a flag, we hand over a pile of context and a key name. In response, we get a value. For many reasons (logging, debugging, etc), we might like to know why we got that value. The spec should provide an API that will allow us to introspect this stuff.

Key info, in my mind, is:

• was there an error? What kind?
• is this the default I provided in my API call or something from the evaluator implementation?

Maybe also:

• which rule lead to this value being chosen?

[toddbaert]

I think we can agree this is a good idea. It's been brought up in a number of threads and in community meetings. Of course, it seems like our ability to surface this data is highly dependent on what is exposed by the underlying provider, but I think defining another set of methods that wrap the flag-value along with some additional evaluation metadata is probably a good approach. This isn't unlike the pattern LaunchDarkly uses, and something like what Split does to surface the additional configuration data associated with a flag.

The trouble is that I don't think all the providers have this class of functionality, and the majority also don't seem to throw exceptions/errors (a perfectly understandable choice in the case of feature flags) further obscuring things.

I'll take a research action item from this, and I'd also appreciate input from anyone familiar with how we might get such information from the various provider SDKs.

[beeme1mr]

This may be a situation where we have to determine what information would be the most valuable and leave it up to the providers to handle. We would then have to handle the situation where this information is not provided but I think that's a reasonable compromise.

[toddbaert]

Relevant PR to this discussion: https://github.com/open-feature/sdk-research/pull/22

[toddbaert]

We've had some interesting discussion about this, see thread here.

In summary, I think we need to decide whether or not we want OpenFeature to rigorously define a set of reasons and force providers to map their own reasons onto that, or allow reasons to be free-form strings. The clincher here, as @beeme1mr points out, is really about whether we expect Application Authors or Integrators to write logic based on these values, or otherwise, they're only important for diagnostics/telemetry/troubleshooting... If we think code will be written around these reason values, I think we should define them well within OpenFeature, and accept that there will be some loss of fidelity between the provider's "reason" and what's exposed in OpenFeature.

@justinabrahms since you opened this discussion, I'm particularly interested in your take.

[justinabrahms]

@toddbaert I'm generally fine with the string implementation here. In my own implementations, I just want that information for logging & telemetry. I can add any simplification or bucketing of those error codes in my hooks that handle that.

[toddbaert]

I'm going to close this discussion for now in an attempt to keep things clean. We have an initial proposal about this in the spec. Of course, we can open PRs against that, or open new discussions if we need to.