Improve feature flag strategy for release management

[PeterYinusa]

Goal: Make software development easier

Background: Currently we use feature flags as part of our build/feature release process, this enables the team to include incomplete features into master without affecting users as the code path is isolated behind a feature flag. Decoupling the development from the access of the feature gives us the ability to carry out controlled testing of the feature.

Problem: As soon as a feature flag is introduced it adds testing complexity as their are additional code paths to test. Currently all the feature flags are configured at compile time with environment variables as opposed to run time. An incomplete list of these environment variables can be found here:

1. Testing features that live behind a feature flag is time consuming

   • Rebuild and install the extension each time we want to test a new feature e.g. onboarding, nfts, token allowance
   • Prep-build-test-* jobs take ~15mins on CI, see prep-build-test,prep-build-test-flask and prep-build-test-mv3

2. Writing e2e tests for features that live behind a feature flag requires a lot of effort, all the tasks below would need to be reviewed, approved and merged.

   • Create a new CI build job e.g. prep-build-test-onboarding, we don't want this list to grow into the following

• Create a new CI test job e.g. test-e2e-chrome-onboarding & test-e2e-firefox-onboarding, we don't want this list to grow into the following

Solution: We want to be able to build and install the extension (debug and/or test version only) once, then dynamically turn on the features we want to test.

Mark: I agree that we're under-utilizing runtime flags. We should create a dev-only settings page for some of these toggles.

Benefits:

• QAs save time as they no longer need to rebuild and install the extension each time they want to test a different feature
• Reduce CI pipeline time (job duration), complexity (list of jobs) and resources (docker images) as we do not need to introduce multiple jobs (build, test) for each feature

[PeterYinusa]

Brad Decker: I have an idea that I think we should explore. Instead of adding new builds like we did for the multichain work to our CI pipeline, can we possibly have a build type for in-progress features and just add the features that are in development to that build type so that if we have multiple big chunks of work in progress that they can be tested together on that build type? It would reduce a lot of effort of duplicating pipelines when kicking off feature development and would prevent issues where multiple big chunks of work land together and cause issues because they'd be under test together. Just an idea, curious to hear your thoughts

[brad-decker]

Also, are flask and beta intended to be parallel channels or a waterfall? Do we always ship to Flask then to Beta then to Main? Is Flask supposed to be feature-complete when releases go out? or should Flask just be where we do what I said in Peter's copied message above?