OpenWebAdvocacy / OpenWebCompetitionPlatform

Open Web Competition Platform is a repo for documenting and tracking issues related to browser and web competition. Its aim is to facilitate coordination between browser vendors, operating systems, developers, and industry to help document outstanding competition issues with an eye towards resolving them more quickly.
MIT License
5 stars 1 forks source link

iOS engine A/B testing #8

Open RByers opened 9 months ago

RByers commented 9 months ago

Description

The Chromium project relies on in-the-wild A/B testing to evaluate the real-world impact of any change we make. It's common, for example, that we find a feature that improves Chrome UI causes performance degradation in an unexpected way, or that a change not intended to be user-visible results in significantly increased crashes. Chrome launches nearly all major features on up to a random 1% sample of our users before enabling it for everyone, and we've found this to be essential in product quality. In addition, the Chromium open source project (on which Chrome is based) has a strict policy of "flag guarding" major and risky changes such that they can be disabled in the wild without pushing a new version of the application.

Apple's proposed design for alternative browser engines in the EU includes a rule that your app must:

Be a separate binary from any app that uses the system-provided web browser engine

Leaving all product issues aside and focusing just on the engineering perspective, this appears to effectively prohibit the use of A/B testing and flag guarding, which are Chrome's primary tools for ensuring product quality. Without such a mechanism, it is essentially guaranteed that a Chrome product using an alternative browser engine would be significantly lower quality and more frustrating to users than one using only the system-provided engine.

Outcome

iOS to support a mechanism for Chrome in the EU to switch transparently (at least on application startup) between an alternative browser engine and the system provided one.