Agenda 2023-10-24

[jgraham]

• Android Emulator Progress
  • Firefox Android is expected to work now, but need to wait for next daily run
  • Planning Chrome for Android runs?
• iOS?
• Score update?
  • From the roadmap 4 is definitely done for Firefox and 5 might be done. For Safari we may still be at stage 3? For Chrome it depends on exact goals.

[jcscottiii]

Attendees: @jgraham @past @gsnedders @jcscottiii

• Android Emulator Progress
  • Firefox Android is expected to work now, but need to wait for next daily run
  • jgraham: This PR should fix that. But waiting on tests to run tomorrow.
  • Planning Chrome for Android runs?
  • jgraham: What's the plan for chrome? Philip had thought Google was to still use the internal infra
  • panos: Our plan was to move to the shared upstream infra. Will discuss internally. Question to jgraham: do you know the level of effort for us. Our plan was for November.
  • jgraham: Should be trivial and should be done by the end of year if started by November.
  • sam: how frequently do we plan on doing the runs?
  • jgraham: Daily at the moment. Let's start there and assess the quality of the data. Next up is to have a discussion on how to incorporate the data into the interop score
  • sam: We should have that discussion with the broader interop audience.
  • jgraham: The first thing we should do is have a mobile version of the dashboard that says BETA on it. Or has some disclaimer that the mobile scores are not official. It'll be something we setup during 2024.
  • sam: The implementation will be a smaller amount of work. The bigger question is the policy aspect.
  • panos: what's the current thing in the wpt tree
  • jgraham: it's gecko nightly runs using gecko view test runner rather than actual firefox browser. Proper firefox mobile would be hard due to the human interaction and things that need to be turned
  • panos: trying to figure out what to do for the chrome side
  • jgraham: do whatever is easier to get working. People rarely look at stable results on the Interop dashboard. Nightly builds maps closely to what people want to see. We know that Safari and Webkit runner has problems with that since it does not exist for mobile
  • iOS?
  • sam: Still having budget discussion internally. Largely from a capacity point of view
  • panos: During BlinkOn, I talked to Microsoft folks. They promised to get back to me on the capacity
  • sam: Things has been stable since September.
  • panos: Any backup plans?
  • sam: There are a few plans. Unclear on which one is the clearest at the moment. Motivated by trying to get more up to date Mac OS images
  • panos: End of year confidence?
  • sam: Pretty low. Would like to have a plan internally on where the runs should be happening. In the future tidying things up on a bunch of infra to make sure we can actually run the runner on iOS.
• Score update?
  • From the roadmap 4 is definitely done for Firefox and 5 might be done. For Safari we may still be at stage 3? For Chrome it depends on exact goals.
  • jgraham: Dashboard still at 40%
  • jcscottiii: Will make a PR to 60% as agreed upon during the Sept 26 meeting
  • panos: What should the new number be with the Firefox work?
  • jgraham: It should go up slightly
  • panos: 70%
  • jgraham: that works especially if the results come in. End of the year, we expect 80-85%

Other notes:

• graham: Seems like we started to discuss things for next year
• sam: Do we have carry over issues?
• sam: No we do not.
• jgraham: Will do this for mobile testing. Will file an issue to let people know this is where we are and what the extra scope is needed. We expect Firefox and Chrome results. Do not anticipate Safari results. Questions on how to incorporate mobile scores into the Interop dashboard.