In our early experimentation with automated results collection for VoiceOver and Safari, we've observed some cases where Safari becomes unresponsive and VoiceOver reports the following AT response for all subsequent tests:
We have elected to deploy the system despite these occasional failures because Test Admins can operate around them by requesting new collection jobs. Doing so also gives us a better opportunity to measure how often the problem occurs--a piece of information which should inform our prioritization for diagnosing and fixing the underlying problem.
In our early experimentation with automated results collection for VoiceOver and Safari, we've observed some cases where Safari becomes unresponsive and VoiceOver reports the following AT response for all subsequent tests:
(example job in GitHub Actions)
We have elected to deploy the system despite these occasional failures because Test Admins can operate around them by requesting new collection jobs. Doing so also gives us a better opportunity to measure how often the problem occurs--a piece of information which should inform our prioritization for diagnosing and fixing the underlying problem.