Mobile Testing Evaluation

[jgraham]

What's been accomplished

• Assessed various options for running mobile testing and settled on using Android Emulator for Chrome / Firefox and iOS Simulator for Safari
• Configured Taskcluster machines to allow nested virtualization to enable running Android Emulator in CI.
• Updated wpt code / infra to allow launching android emulator and installing packages.
• Added support for installing nightly geckoview in the wpt frontend, so that wpt run firefox_android works.
• Added infra job for `firefox_android.
• Added daily firefox_android runs based on nightly geckoview builds.
• PoC work for running stable Safari iOS builds in the iOS simulator.

What's expected to be complete by the EoY

• Support for installing and running Chrome Android builds in wpt
• Chrome Android daily runs (assuming that there's not a decision to prefer the existing runs based on Google internal infrastructure).
• A clear plan on what infrastructure to use for Safari/WebKit going forward.

What further work is required

Most of the infrastructure side work is expected to be complete, although some further work could be required (very likely Safari/WebKit, maybe some smaller cleanup e.g. if we require stable results for browsers that currently only have nightly work).

The main work going forward will be for the Interop team to decide how to incorporate mobile results into future metrics (if at all). Assuming mobile results will be present in some way, work on the scoring and frontend will be required to match the requirements of the Interop group.

[jgraham]

CC @gsnedders @past @jcscottiii @WeizhongX