Feature Toggles - Decouple releasing and deployment using release\feature toggles to help provide better QA and UX testing with real users and data

[rianfowler]

User Problem:

VSA Teams are do not have a way to perform QA or UAT testing with actual data\users without deploying their application into production. Once it is pushed to production (occurs with daily deployment that they don't control) all users can access it even if if the application was not ready yet.

Epic #1326 - Provides more details about the problems with our staging environment not having actual users or real data.

Details

At a high level, we want to have the a couple of classifications of control:

• platform: mandatory support in every app. anyone in an operations capacity can utilize these flags to manage the general stability of the website without specific business knowledge / context about the app.
  • shut an app completely off
  • pause an app / notify of degraded service
  • rollback an app to a previous version of feature flags
• application: generally application teams should be able to gate features in the way that best fits their needs. The platform documents and encourages best practices for these common use cases of application level feature control:
  • percentage based rollout
  • a/b testing js features / content including related analytics (e.g. goal setting / tracking tied to a/b tests)
  • runtime feature flag changes

Pain Points: VFS Teams - Feature Toggles are much easier to configure

Status:

• Discovery of available Tools (LaunchDarkly, unleash, Flipper)
• Team decided on Flipper - https://github.com/department-of-veterans-affairs/va.gov-team/issues/775
• Added the ability for VSA teams to leverage Flipper to create feature\release toggles.
• Added the ability for VSA teams to allow only a % of users to be affected by a feature\release toggle to help support better QA efforts, beta and UAT testing.
• Adding basic user management and logging to the Flipper Admin Portal so we can better regulate and track "who" toggled "what" and "when" - #1764

[rianfowler]

Problem

• releases are tied to deploys / no runtime feature management
• features are gated to environments (no production like environment to test on)

Goal

• support feature flag driven development, testing in production, percentage based rollouts, a/b testing, continuous improvement, continuous delivery

Notes

Operations control of features is an important goal of the VA.gov platform. Currently features can be gated by environment (dev, staging, production). This approach isn't well suited for continuous integration / continuous delivery:

• there's no way to know what features are gated to an environment without reading the source code (e.g. no audit trail)
• changes are not validated with post-release telemetry (e.g. a/b testing)
• altering feature flags requires a full build / deploy

At a high level, we want to have the a couple of classifications of control:

• platform: mandatory support in every app. anyone in an operations capacity can utilize these flags to manage the general stability of the website without specific business knowledge / context about the app.
  • shut an app completely off
  • pause an app / notify of degraded service
  • rollback an app to a previous version of feature flags
• application: generally application teams should be able to gate features in the way that best fits their needs. The platform documents and encourages best practices for tgese common use cases of application level feature control:
  • percentage based rollout
  • a/b testing js features / content including related analytics (e.g. goal setting / tracking tied to a/b tests)
  • runtime feature flag changes

COTS / frameworks

• launch darkly
  • high end hosted solution
  • team management
  • live feature flag updates
  • frameworks / apis
• Google optimize
  • integrated solution for google analytics
  • can get to the variant information but no frameworks built out
  • focused on a/b testing more than general feature flag management
• app knobs
  • lower end hosted solution
• unleash
  • roll your own feature flag api
  • has some dashboards / api server features
• Optimizely
• Flipper
  • maintainer has offered to help us

Commenting on approach

This epic represents a request for comments on feature flag approach.

• comment on the goal- I listed a few processes we could focus on supporting. Would like to get this down to just one or two. e.g. that platform support feature flag driven development. our user centered design process and analytics align to that goal in the following ways.
• comment on the grouping of control by operations vs app management concerns
• comment on the PR that demo's launch darkly
• comment on the third party offerings (e.g. add more details or additional frameworks to evaluate)

Tasks

• [x] Evaluate launch darkly - issue
• [x] Evaluate flipper - issue
• [x] Evaluate unleash issue
• [ ] rollout plan

Info

• https://www.martinfowler.com/articles/feature-toggles.html
• http://featureflags.io

[brandonrapp]

Next Steps: Discovery work is needed to work to identify where the pain points are in terms of using this.