[Research] Tracking functionality

[toddbaert]

Many SDKs have functionality to "track" specific user actions. These typically seem to be methods that can be called to record that a certain flag-related event has taken place - often some goal related to a business outcome (though this paradigm might not be universally true for all implementations).

Examples:

https://help.split.io/hc/en-us/articles/360020585772-Events https://docs.launchdarkly.com/sdk/features/events#tracking-events

It should be noted that some vendors don't have the functionality, so we will want to consider that in our implementation. If we decide to make this sort of tracking the business of OpenFeature, we may want to allow for registering a separate tracking provider in such cases.

cc @weyert

Relates to: https://github.com/open-feature/spec/pull/102

[weyert]

PostHog supports the following:

posthog.capture('[event-name]', {property1: 'value', property2: 'another value'});

See: https://posthog.com/docs/integrate/client/js#capture

[weyert]

Optimizily appears to support something similar:

var attributes = { logged_in: true };
var user = optimizelyClient.createUserContext('user123', attributes);

var tags = {
  category: 'shoes',
  purchase_count: 2
};

user.trackEvent("my_purchase_event_key", tags);

See: https://docs.developers.optimizely.com/experimentation/v4.0.0-full-stack/docs/track-events

[weyert]

FlagShip.io appears to support some sort of track events via:

import { HitType, EventCategory } from "@flagship.io/js-sdk";

visitor.sendHit({
  type: HitType.EVENT, //or "EVENT"
  category: EventCategory.USER_ENGAGEMENT, // or EventCategory.ACTION_TRACKING
  action: "click",
  label: "label",
  value: 100,
});

See: https://docs.developers.flagship.io/docs/js-v3#hit-tracking

[weyert]

DevCycle supports tracking events:

const event = {
    type: 'my_event_type', // this is required
    date: new Date(),
    target: 'my_target',
    value: 5,
    metaData: {
        key: 'value'
    }
}
dvcClient.track(event)
dvcClient.flushEvents()

See: https://docs.devcycle.com/docs/sdk/client-side-sdks/javascript#tracking-events

[beeme1mr]

Hey @weyert, would you be willing to put together an OFEP for adding tracking events to the spec?

[weyert]

Sure :)

[justinabrahms]

I think this is the same as https://github.com/open-feature/spec/issues/78 ?

[beeme1mr]

Yes it's the same thing. I'll close the one in the spec in favor of the research issue. The spec can be updated once an OpenFeature Enhancement Proposal has been accepted.

[InTheCloudDan]

I came across this repo: https://github.com/davidwells/analytics which is for a similar use case it seems and might be able to provide inspiration.

[powelli]

Not sure about other providers, but with Optimizely at least a tracking event/impression can be recorded each time a flag evaluation is carried out: https://support.optimizely.com/hc/en-us/articles/4410284241165#Full_Stack

The firing of these events can be controlled as part of the client SDK initialisation, and via additional context passed to the evaluation method: https://docs.developers.optimizely.com/full-stack/v4.0/docs/decide-methods-java#optimizelydecideoption

This is useful in some use-cases we have as costs are based on the volume of tracked events, but some evaluations don't need to be tracked (e.g. if they're not associated with an experiment).

Realise that Optimizely aren't on board as a provider yet, but when considering this tracking enhancement it would be useful to consider if these per-decision controls are something that could be supported by OpenFeature where the provider supports it.

[beeme1mr]

Hey @powelli, an Optimizely provider should be able to implement impression tracking right now. The initialization of the provider could control what impressions are collected. The only part current missing from most SDKs (but coming soon) is a shutdown handler. This will be called when an application shutdowns and would allow a provider developer to explicitly flush events or close open handlers.

This issue was focused on tracking events outside of impressions. For example, Optimizely allows you to track events. This is primary used in experimentation and is not collected automatically as part of a typical flag evaluation.

[justinabrahms]

We currently use hook hints to control tracking. By default, I believe we're doing something like impression tracking mentioned above via provider hooks, but it can be temporarily turned off with a hookhint (good for eager evaluation).

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in next 60 days.

[justinabrahms]

@s-sen - This may be an issue y'all would like to get involved with.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in next 60 days.

[weyert]

I am going to pick up this issue again. Interested to sort something out.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in next 60 days.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in next 60 days.

[jrydberg]

A few vendors (segment.io, Rudderstack, Hightouch) have standardized on this specification:

• https://segment.com/docs/connections/spec/

One good option would be to adopt and make an open version of the spec. Either as first-class citizen in OpenFeature or as a sister project.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in next 60 days.

[tapico-weyert]

What do you mean with 'One good option would be to adopt and make an open version of the spec.'?

Do you mean a OpenFeature spec or more separate from OpenFeature? Or its discussed in sibling spec in the OpenFeature repo?

[weyert]

The standard looks alright but I think it has a bit awkward wording (e.g. traits) in the context of OpenFeature maybe we can adopt the idea but change the schema.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in next 60 days.