search

web-platform-tests / wpt

Test suites for Web platform specs — including WHATWG, W3C, and others
https://web-platform-tests.org/
Other
4.93k stars 3.07k forks source link

Safari runs are often missing since June 20th, 2024 #46899

Open jrobbins opened 3 months ago

jrobbins commented 3 months ago

There have been no stable Safari runs recorded for the past four days: https://wpt.fyi/runs?label=master&label=stable&max-count=100&product=chrome&product=firefox&product=safari

CC @gsnedders

DanielRyanSmith commented 2 months ago

It looks like we've received 3 runs since then, but the results are only coming in intermittently. Additionally, experimental Safari runs are missing from July 5th-July 12th (https://wpt.fyi/runs?label=master&label=experimental&max-count=100&product=chrome&product=edge&product=firefox&product=safari&product=webkitgtk)

past commented 1 month ago

We've been getting some Safari runs occasionally. @gsnedders is working on it.

gsnedders commented 1 month ago

This also applies to STP.

This is effectively a regression from #46737 (migrating to macOS Sonoma), and we're seeing comparable Azure Pipelines issues to what we saw last year after migrating to macOS Ventura (https://github.com/web-platform-tests/wpt/issues/40085).

Downgrading to macOS Ventura isn't an option, as it is no longer supported by Safari Technology Preview.

Given the many-months delay last year for Azure Pipelines to resolve any of this, and the reduction in Azure Pipelines capacity from next month, with ongoing work to migrate away from Azure Pipelines before then, there's likely little we can do except "hit the 'rerun failed builds' button".