microsoft / accessibility-insights-web

Accessibility Insights for Web
https://accessibilityinsights.io
MIT License
839 stars 150 forks source link

Ongoing issues with assessment getting stuck in "scanning" mode #5376

Closed KJohansonMicrosoft closed 1 year ago

KJohansonMicrosoft commented 2 years ago

I frequently have issues where my assessments get stuck in “scanning” mode (see below), and I have to close the assessment windows and reopen to continue.

I don't know how to reproduce, as this issue shows up randomly, not on all parts of an assessment, or all assessments.

I expected to be able to specify whether a page passed or failed that specific assessment.

image

Context

Edition Windows 10 Enterprise Version 21H2 Installed on ‎10/‎14/‎2021 OS build 19044.1645 Experience Windows Feature Experience Pack 120.2212.4170.0

Microsoft Edge Version 100.0.1185.39 (Official build) (64-bit)

Are you willing to submit a PR?

Target page: https://www.microsoft.com/en-us/licensing/learn-more/trainings?activetab=licensing-call-tab%3aprimaryr3

I don't know how to submit a PR

Did you search for similar existing issues?

Yes

Additional context

N/A

jlperkins commented 2 years ago

Hello! Thanks for letting us know about this issue. Can you give us any more information so that we might be able to reproduce it? For example, if there is anything you are typically working on when it hangs. Can you think of anything specific that you were doing right before this issue typically happens?

ghost commented 2 years ago

The team requires additional author feedback; please review their replies and update this issue accordingly. Thank you for contributing to Accessibility Insights!

KJohansonMicrosoft commented 2 years ago

Not really. I haven't been documenting which parts of the assessment trigger this problem. But I see the "scanning" icon as soon as I click on a new segment of the assessment.

jlperkins commented 2 years ago

Gotcha. Without a consistent or reproducible case this might be tricky to triage... what is the exact version of the extension which you are running? (Version is visible at the bottom left of the popup) image

ghost commented 2 years ago

The team requires additional author feedback; please review their replies and update this issue accordingly. Thank you for contributing to Accessibility Insights!

jlperkins commented 2 years ago

I was able to reproduce both this issue and the one you emailed about (text included below) by switching quickly between different assisted requirements. Thank you for pointing out this issue, we will work to fix it!

Issue raised via email:

I am conducting accessibility training on the Licensing Briefs page, and the visual helper slider is not working on the Visible focus 3.1 assessment. I have encountered this one or two other times in recent weeks, but I don’t recall whether the problem occurred on the same segment within the assessment. As with the other issue I reported this morning, nothing except closing the assessment and restarting from the same spot within the assessment helped. Refreshing the page had no impact.

ghost commented 2 years ago

This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!

KJohansonMicrosoft commented 2 years ago

Version 2.31.1

dbjorge commented 2 years ago

For the "visual helper cannot be toggled on" variant, I captured the following logs from the background page during the repro:

full background logs ``` Unable to interpret message - Object { messageType: "insights/TabStopsViewStore/state/current", tabId: 161 } Unable to interpret message - Object { messageType: "insights/TabStopsViewStore/state/current", tabId: 161 } Unable to interpret message - Object { messageType: "insights/TabStopsViewStore/state/current", tabId: 161 } sendMessageToTab(161) failed for message {"isStoreUpdateMessage":true,"storeId":"AssessmentStore","messageType":"STORE_CHANGED","storeType":0,"payload":{"persistedTabInfo":null,"assessmentNavState":{"selectedTestType":24,"selectedTestSubview":"area-alt","expandedTestType":24},"assessments":{"automated-checks":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"area-alt":{"status":1,"id":"area-alt","instances":[]},"aria-allowed-attr":{"status":1,"id":"aria-allowed-attr","instances":[]},"aria-command-name":{"status":1,"id":"aria-command-name","instances":[]},"aria-hidden-body":{"status":1,"id":"aria-hidden-body","instances":[]},"aria-hidden-focus":{"status":1,"id":"aria-hidden-focus","instances":[]},"aria-input-field-name":{"status":1,"id":"aria-input-field-name","instances":[]},"aria-meter-name":{"status":1,"id":"aria-meter-name","instances":[]},"aria-progressbar-name":{"status":1,"id":"aria-progressbar-name","instances":[]},"aria-required-attr":{"status":1,"id":"aria-required-attr","instances":[]},"aria-required-children":{"status":1,"id":"aria-required-children","instances":[]},"aria-required-parent":{"status":1,"id":"aria-required-parent","instances":[]},"aria-roledescription":{"status":1,"id":"aria-roledescription","instances":[]},"aria-roles":{"status":1,"id":"aria-roles","instances":[]},"aria-toggle-field-name":{"status":1,"id":"aria-toggle-field-name","instances":[]},"aria-tooltip-name":{"status":1,"id":"aria-tooltip-name","instances":[]},"aria-valid-attr-value":{"status":1,"id":"aria-valid-attr-value","instances":[]},"aria-valid-attr":{"status":1,"id":"aria-valid-attr","instances":[]},"audio-caption":{"status":1,"id":"audio-caption","instances":[]},"autocomplete-valid":{"status":1,"id":"autocomplete-valid","instances":[]},"avoid-inline-spacing":{"status":1,"id":"avoid-inline-spacing","instances":[]},"blink":{"status":1,"id":"blink","instances":[]},"button-name":{"status":1,"id":"button-name","instances":[]},"bypass":{"status":1,"id":"bypass","instances":[]},"color-contrast":{"status":1,"id":"color-contrast","instances":[]},"definition-list":{"status":1,"id":"definition-list","instances":[]},"dlitem":{"status":1,"id":"dlitem","instances":[]},"document-title":{"status":1,"id":"document-title","instances":[]},"duplicate-id-active":{"status":1,"id":"duplicate-id-active","instances":[]},"duplicate-id-aria":{"status":1,"id":"duplicate-id-aria","instances":[]},"frame-focusable-content":{"status":1,"id":"frame-focusable-content","instances":[]},"frame-title":{"status":1,"id":"frame-title","instances":[]},"html-has-lang":{"status":1,"id":"html-has-lang","instances":[]},"html-lang-valid":{"status":1,"id":"html-lang-valid","instances":[]},"html-xml-lang-mismatch":{"status":1,"id":"html-xml-lang-mismatch","instances":[]},"image-alt":{"status":1,"id":"image-alt","instances":[]},"input-button-name":{"status":1,"id":"input-button-name","instances":[]},"input-image-alt":{"status":1,"id":"input-image-alt","instances":[]},"label":{"status":1,"id":"label","instances":[]},"link-name":{"status":1,"id":"link-name","instances":[]},"list":{"status":1,"id":"list","instances":[]},"listitem":{"status":1,"id":"listitem","instances":[]},"marquee":{"status":1,"id":"marquee","instances":[]},"meta-refresh":{"status":1,"id":"meta-refresh","instances":[]},"nested-interactive":{"status":1,"id":"nested-interactive","instances":[]},"object-alt":{"status":1,"id":"object-alt","instances":[]},"role-img-alt":{"status":1,"id":"role-img-alt","instances":[]},"select-name":{"status":1,"id":"select-name","instances":[]},"server-side-image-map":{"status":1,"id":"server-side-image-map","instances":[]},"svg-img-alt":{"status":1,"id":"svg-img-alt","instances":[]},"td-headers-attr":{"status":1,"id":"td-headers-attr","instances":[]},"th-has-data-cells":{"status":1,"id":"th-has-data-cells","instances":[]},"valid-lang":{"status":1,"id":"valid-lang","instances":[]},"video-caption":{"status":1,"id":"video-caption","instances":[]}},"testStepStatus":{"area-alt":{"stepFinalResult":1,"isStepScanned":false},"aria-allowed-attr":{"stepFinalResult":1,"isStepScanned":false},"aria-command-name":{"stepFinalResult":1,"isStepScanned":false},"aria-hidden-body":{"stepFinalResult":1,"isStepScanned":false},"aria-hidden-focus":{"stepFinalResult":1,"isStepScanned":false},"aria-input-field-name":{"stepFinalResult":1,"isStepScanned":false},"aria-meter-name":{"stepFinalResult":1,"isStepScanned":false},"aria-progressbar-name":{"stepFinalResult":1,"isStepScanned":false},"aria-required-attr":{"stepFinalResult":1,"isStepScanned":false},"aria-required-children":{"stepFinalResult":1,"isStepScanned":false},"aria-required-parent":{"stepFinalResult":1,"isStepScanned":false},"aria-roledescription":{"stepFinalResult":1,"isStepScanned":false},"aria-roles":{"stepFinalResult":1,"isStepScanned":false},"aria-toggle-field-name":{"stepFinalResult":1,"isStepScanned":false},"aria-tooltip-name":{"stepFinalResult":1,"isStepScanned":false},"aria-valid-attr-value":{"stepFinalResult":1,"isStepScanned":false},"aria-valid-attr":{"stepFinalResult":1,"isStepScanned":false},"audio-caption":{"stepFinalResult":1,"isStepScanned":false},"autocomplete-valid":{"stepFinalResult":1,"isStepScanned":false},"avoid-inline-spacing":{"stepFinalResult":1,"isStepScanned":false},"blink":{"stepFinalResult":1,"isStepScanned":false},"button-name":{"stepFinalResult":1,"isStepScanned":false},"bypass":{"stepFinalResult":1,"isStepScanned":false},"color-contrast":{"stepFinalResult":1,"isStepScanned":false},"definition-list":{"stepFinalResult":1,"isStepScanned":false},"dlitem":{"stepFinalResult":1,"isStepScanned":false},"document-title":{"stepFinalResult":1,"isStepScanned":false},"duplicate-id-active":{"stepFinalResult":1,"isStepScanned":false},"duplicate-id-aria":{"stepFinalResult":1,"isStepScanned":false},"frame-focusable-content":{"stepFinalResult":1,"isStepScanned":false},"frame-title":{"stepFinalResult":1,"isStepScanned":false},"html-has-lang":{"stepFinalResult":1,"isStepScanned":false},"html-lang-valid":{"stepFinalResult":1,"isStepScanned":false},"html-xml-lang-mismatch":{"stepFinalResult":1,"isStepScanned":false},"image-alt":{"stepFinalResult":1,"isStepScanned":false},"input-button-name":{"stepFinalResult":1,"isStepScanned":false},"input-image-alt":{"stepFinalResult":1,"isStepScanned":false},"label":{"stepFinalResult":1,"isStepScanned":false},"link-name":{"stepFinalResult":1,"isStepScanned":false},"list":{"stepFinalResult":1,"isStepScanned":false},"listitem":{"stepFinalResult":1,"isStepScanned":false},"marquee":{"stepFinalResult":1,"isStepScanned":false},"meta-refresh":{"stepFinalResult":1,"isStepScanned":false},"nested-interactive":{"stepFinalResult":1,"isStepScanned":false},"object-alt":{"stepFinalResult":1,"isStepScanned":false},"role-img-alt":{"stepFinalResult":1,"isStepScanned":false},"select-name":{"stepFinalResult":1,"isStepScanned":false},"server-side-image-map":{"stepFinalResult":1,"isStepScanned":false},"svg-img-alt":{"stepFinalResult":1,"isStepScanned":false},"td-headers-attr":{"stepFinalResult":1,"isStepScanned":false},"th-has-data-cells":{"stepFinalResult":1,"isStepScanned":false},"valid-lang":{"stepFinalResult":1,"isStepScanned":false},"video-caption":{"stepFinalResult":1,"isStepScanned":false}}},"keyboardInteraction":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"keyboard-navigation":{"status":1,"id":"keyboard-navigation","instances":[]},"no-keyboard-traps":{"status":1,"id":"no-keyboard-traps","instances":[]},"on-focus":{"status":1,"id":"on-focus","instances":[]},"on-input":{"status":1,"id":"on-input","instances":[]},"on-keystroke-timing":{"status":1,"id":"on-keystroke-timing","instances":[]},"character-key-shortcuts":{"status":1,"id":"character-key-shortcuts","instances":[]}},"testStepStatus":{"keyboard-navigation":{"stepFinalResult":1,"isStepScanned":false},"no-keyboard-traps":{"stepFinalResult":1,"isStepScanned":false},"on-focus":{"stepFinalResult":1,"isStepScanned":false},"on-input":{"stepFinalResult":1,"isStepScanned":false},"on-keystroke-timing":{"stepFinalResult":1,"isStepScanned":false},"character-key-shortcuts":{"stepFinalResult":1,"isStepScanned":false}}},"visibleFocusOrder":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"visible-focus":{"status":1,"id":"visible-focus","instances":[]},"revealing-content":{"status":1,"id":"revealing-content","instances":[]},"modal-dialogs":{"status":1,"id":"modal-dialogs","instances":[]},"closing-content":{"status":1,"id":"closing-content","instances":[]},"focus-order":{"status":1,"id":"focus-order","instances":[]}},"testStepStatus":{"visible-focus":{"stepFinalResult":1,"isStepScanned":false},"revealing-content":{"stepFinalResult":1,"isStepScanned":false},"modal-dialogs":{"stepFinalResult":1,"isStepScanned":false},"closing-content":{"stepFinalResult":1,"isStepScanned":false},"focus-order":{"stepFinalResult":1,"isStepScanned":false}}},"landmarks":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"landmark-roles":{"status":1,"id":"landmark-roles","instances":[]},"primary-content":{"status":1,"id":"primary-content","instances":[]},"no-repeating-content":{"status":1,"id":"no-repeating-content","instances":[]}},"testStepStatus":{"landmark-roles":{"stepFinalResult":1,"isStepScanned":false},"primary-content":{"stepFinalResult":1,"isStepScanned":false},"no-repeating-content":{"stepFinalResult":1,"isStepScanned":false}}},"headings":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"headingFunction":{"status":1,"id":"headingFunction","instances":[]},"missingHeadings":{"status":1,"id":"missingHeadings","instances":[]},"headingLevel":{"status":1,"id":"headingLevel","instances":[]}},"testStepStatus":{"headingFunction":{"stepFinalResult":1,"isStepScanned":false},"missingHeadings":{"stepFinalResult":1,"isStepScanned":false},"headingLevel":{"stepFinalResult":1,"isStepScanned":false}}},"repetitiveContent":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"bypassBlocks":{"status":1,"id":"bypassBlocks","instances":[]},"consistentNavigation":{"status":1,"id":"consistentNavigation","instances":[]},"consistentIdentification":{"status":1,"id":"consistentIdentification","instances":[]}},"testStepStatus":{"bypassBlocks":{"stepFinalResult":1,"isStepScanned":false},"consistentNavigation":{"stepFinalResult":1,"isStepScanned":false},"consistentIdentification":{"stepFinalResult":1,"isStepScanned":false}}},"linksAssessment":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"linkFunction":{"status":1,"id":"linkFunction","instances":[]},"linkPurpose":{"status":1,"id":"linkPurpose","instances":[]}},"testStepStatus":{"linkFunction":{"stepFinalResult":1,"isStepScanned":false},"linkPurpose":{"stepFinalResult":1,"isStepScanned":false}}},"nativeWidgets":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"widgetFunction":{"status":1,"id":"widgetFunction","instances":[]},"nativeInstructions":{"status":1,"id":"nativeInstructions","instances":[]},"nativeLabel":{"status":1,"id":"nativeLabel","instances":[]},"nativeCues":{"status":1,"id":"nativeCues","instances":[]},"nativeAutocomplete":{"status":1,"id":"nativeAutocomplete","instances":[]}},"testStepStatus":{"widgetFunction":{"stepFinalResult":1,"isStepScanned":false},"nativeInstructions":{"stepFinalResult":1,"isStepScanned":false},"nativeLabel":{"stepFinalResult":1,"isStepScanned":false},"nativeCues":{"stepFinalResult":1,"isStepScanned":false},"nativeAutocomplete":{"stepFinalResult":1,"isStepScanned":false}}},"customWidgets":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"designPattern":{"status":1,"id":"designPattern","instances":[]},"customWidgetsInstructions":{"status":1,"id":"customWidgetsInstructions","instances":[]},"customWidgetsLabel":{"status":1,"id":"customWidgetsLabel","instances":[]},"roleStateProperty":{"status":1,"id":"roleStateProperty","instances":[]},"customWidgetsCues":{"status":1,"id":"customWidgetsCues","instances":[]},"keyboardInteraction":{"status":1,"id":"keyboardInteraction","instances":[]}},"testStepStatus":{"designPattern":{"stepFinalResult":1,"isStepScanned":false},"customWidgetsInstructions":{"stepFinalResult":1,"isStepScanned":false},"customWidgetsLabel":{"stepFinalResult":1,"isStepScanned":false},"roleStateProperty":{"stepFinalResult":1,"isStepScanned":false},"customWidgetsCues":{"stepFinalResult":1,"isStepScanned":false},"keyboardInteraction":{"stepFinalResult":1,"isStepScanned":false}}},"timedEvents":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"time-limits":{"status":1,"id":"time-limits","instances":[]},"moving-content":{"status":1,"id":"moving-content","instances":[]},"auto-updating-content":{"status":1,"id":"auto-updating-content","instances":[]},"audio-control":{"status":1,"id":"audio-control","instances":[]}},"testStepStatus":{"time-limits":{"stepFinalResult":1,"isStepScanned":false},"moving-content":{"stepFinalResult":1,"isStepScanned":false},"auto-updating-content":{"stepFinalResult":1,"isStepScanned":false},"audio-control":{"stepFinalResult":1,"isStepScanned":false}}},"errors":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"errorIdentification":{"status":1,"id":"errorIdentification","instances":[]},"errorSuggestion":{"status":1,"id":"errorSuggestion","instances":[]},"errorPrevention":{"status":1,"id":"errorPrevention","instances":[]},"statusMessages":{"status":1,"id":"statusMessages","instances":[]}},"testStepStatus":{"errorIdentification":{"stepFinalResult":1,"isStepScanned":false},"errorSuggestion":{"stepFinalResult":1,"isStepScanned":false},"errorPrevention":{"stepFinalResult":1,"isStepScanned":false},"statusMessages":{"stepFinalResult":1,"isStepScanned":false}}},"page":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"pageTitle":{"status":1,"id":"pageTitle","instances":[]},"frameTitle":{"status":1,"id":"frameTitle","instances":[]},"generalNavigation":{"status":1,"id":"generalNavigation","instances":[]}},"testStepStatus":{"pageTitle":{"stepFinalResult":1,"isStepScanned":false},"frameTitle":{"stepFinalResult":1,"isStepScanned":false},"generalNavigation":{"stepFinalResult":1,"isStepScanned":false}}},"parsing":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"parsing":{"status":1,"id":"parsing","instances":[]}},"testStepStatus":{"parsing":{"stepFinalResult":1,"isStepScanned":false}}},"images":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"imageFunction":{"status":1,"id":"imageFunction","instances":[]},"textAlternative":{"status":1,"id":"textAlternative","instances":[]},"imageOfText":{"status":1,"id":"imageOfText","instances":[]},"captcha":{"status":1,"id":"captcha","instances":[]}},"testStepStatus":{"imageFunction":{"stepFinalResult":1,"isStepScanned":false},"textAlternative":{"stepFinalResult":1,"isStepScanned":false},"imageOfText":{"stepFinalResult":1,"isStepScanned":false},"captcha":{"stepFinalResult":1,"isStepScanned":false}}},"language":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"languageOfPage":{"status":1,"id":"languageOfPage","instances":[]},"languageOfParts":{"status":1,"id":"languageOfParts","instances":[]},"textDirection":{"status":1,"id":"textDirection","instances":[]}},"testStepStatus":{"languageOfPage":{"stepFinalResult":1,"isStepScanned":false},"languageOfParts":{"stepFinalResult":1,"isStepScanned":false},"textDirection":{"stepFinalResult":1,"isStepScanned":false}}},"color":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"useOfColor":{"status":1,"id":"useOfColor","instances":[]},"sensoryCharacteristics":{"status":1,"id":"sensoryCharacteristics","instances":[]},"auditoryCues":{"status":1,"id":"auditoryCues","instances":[]},"flashing":{"status":1,"id":"flashing","instances":[]}},"testStepStatus":{"useOfColor":{"stepFinalResult":1,"isStepScanned":false},"sensoryCharacteristics":{"stepFinalResult":1,"isStepScanned":false},"auditoryCues":{"stepFinalResult":1,"isStepScanned":false},"flashing":{"stepFinalResult":1,"isStepScanned":false}}},"textLegibility":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"highContrastMode":{"status":1,"id":"highContrastMode","instances":[]},"resizeText":{"status":1,"id":"resizeText","instances":[]},"contrast":{"status":1,"id":"contrast","instances":[]},"orientation":{"status":1,"id":"orientation","instances":[]},"reflow":{"status":1,"id":"reflow","instances":[]},"textSpacing":{"status":1,"id":"textSpacing","instances":[]},"hoverFocusContent":{"status":1,"id":"hoverFocusContent","instances":[]}},"testStepStatus":{"highContrastMode":{"stepFinalResult":1,"isStepScanned":false},"resizeText":{"stepFinalResult":1,"isStepScanned":false},"contrast":{"stepFinalResult":1,"isStepScanned":false},"orientation":{"stepFinalResult":1,"isStepScanned":false},"reflow":{"stepFinalResult":1,"isStepScanned":false},"textSpacing":{"stepFinalResult":1,"isStepScanned":false},"hoverFocusContent":{"stepFinalResult":1,"isStepScanned":false}}},"audioVideoOnly":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"audioOnlyEquivalent":{"status":1,"id":"audioOnlyEquivalent","instances":[]},"videoOnlyEquivalent":{"status":1,"id":"videoOnlyEquivalent","instances":[]}},"testStepStatus":{"audioOnlyEquivalent":{"stepFinalResult":1,"isStepScanned":false},"videoOnlyEquivalent":{"stepFinalResult":1,"isStepScanned":false}}},"prerecordedMultimedia":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"captions":{"status":1,"id":"captions","instances":[]},"noObstruction":{"status":1,"id":"noObstruction","instances":[]},"audioDescription":{"status":1,"id":"audioDescription","instances":[]},"synchronization":{"status":1,"id":"synchronization","instances":[]},"noConflict":{"status":1,"id":"noConflict","instances":[]}},"testStepStatus":{"captions":{"stepFinalResult":1,"isStepScanned":false},"noObstruction":{"stepFinalResult":1,"isStepScanned":false},"audioDescription":{"stepFinalResult":1,"isStepScanned":false},"synchronization":{"stepFinalResult":1,"isStepScanned":false},"noConflict":{"stepFinalResult":1,"isStepScanned":false}}},"liveMultimedia":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"liveCaptions":{"status":1,"id":"liveCaptions","instances":[]}},"testStepStatus":{"liveCaptions":{"stepFinalResult":1,"isStepScanned":false}}},"sequence":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"cssPositioning":{"status":1,"id":"cssPositioning","instances":[]},"layoutTables":{"status":1,"id":"layoutTables","instances":[]},"columns":{"status":1,"id":"columns","instances":[]}},"testStepStatus":{"cssPositioning":{"stepFinalResult":1,"isStepScanned":false},"layoutTables":{"stepFinalResult":1,"isStepScanned":false},"columns":{"stepFinalResult":1,"isStepScanned":false}}},"semanticsAssessment":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"cssContent":{"status":1,"id":"cssContent","instances":[]},"tableSemantics":{"status":1,"id":"tableSemantics","instances":[]},"headers":{"status":1,"id":"headers","instances":[]},"headersAttribute":{"status":1,"id":"headersAttribute","instances":[]},"lists":{"status":1,"id":"lists","instances":[]},"emphasis":{"status":1,"id":"emphasis","instances":[]},"quotes":{"status":1,"id":"quotes","instances":[]},"letterSpacing":{"status":1,"id":"letterSpacing","instances":[]}},"testStepStatus":{"cssContent":{"stepFinalResult":1,"isStepScanned":false},"tableSemantics":{"stepFinalResult":1,"isStepScanned":false},"headers":{"stepFinalResult":1,"isStepScanned":false},"headersAttribute":{"stepFinalResult":1,"isStepScanned":false},"lists":{"stepFinalResult":1,"isStepScanned":false},"emphasis":{"stepFinalResult":1,"isStepScanned":false},"quotes":{"stepFinalResult":1,"isStepScanned":false},"letterSpacing":{"stepFinalResult":1,"isStepScanned":false}}},"pointerMotion":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"pointer-gestures":{"status":1,"id":"pointer-gestures","instances":[]},"pointer-cancellation":{"status":1,"id":"pointer-cancellation","instances":[]},"motion-operation":{"status":1,"id":"motion-operation","instances":[]}},"testStepStatus":{"pointer-gestures":{"stepFinalResult":1,"isStepScanned":false},"pointer-cancellation":{"stepFinalResult":1,"isStepScanned":false},"motion-operation":{"stepFinalResult":1,"isStepScanned":false}}},"contrast":{"fullAxeResultsMap":null,"generatedAssessmentInstancesMap":null,"manualTestStepResultMap":{"ui-components":{"status":1,"id":"ui-components","instances":[]},"state-changes":{"status":1,"id":"state-changes","instances":[]},"graphics":{"status":1,"id":"graphics","instances":[]}},"testStepStatus":{"ui-components":{"stepFinalResult":1,"isStepScanned":false},"state-changes":{"stepFinalResult":1,"isStepScanned":false},"graphics":{"stepFinalResult":1,"isStepScanned":false}}}},"resultDescription":""}} with browser error message: Cannot read properties of undefined (reading 'scanning') sendMessageToTab(161) failed for message {"isStoreUpdateMessage":true,"storeId":"VisualizationStore","messageType":"STORE_CHANGED","storeType":1,"payload":{"tests":{"adhoc":{"headings":{"enabled":false},"issues":{"enabled":false},"landmarks":{"enabled":false},"tabStops":{"enabled":false},"color":{"enabled":false},"needsReview":{"enabled":false}},"assessments":{"headingsAssessment":{"enabled":false,"stepStatus":{}},"colorSensoryAssessment":{"enabled":false,"stepStatus":{}},"languageAssessment":{"enabled":false,"stepStatus":{}},"landmarksAssessment":{"enabled":false,"stepStatus":{}},"pageAssessment":{"enabled":false,"stepStatus":{}},"repetitiveContentAssessment":{"enabled":false,"stepStatus":{}},"keyboardInteractionAssessment":{"enabled":false,"stepStatus":{"keyboard-navigation":false}},"audioVideoOnlyAssessment":{"enabled":false,"stepStatus":{}},"errorsAssessment":{"enabled":false,"stepStatus":{}},"timedEventsAssessment":{"enabled":false,"stepStatus":{}},"parsingAssessment":{"enabled":false,"stepStatus":{}},"prerecordedMultimediaAssessment":{"enabled":false,"stepStatus":{}},"liveMultimediaAssessment":{"enabled":false,"stepStatus":{}},"visibleFocusOrderAssessment":{"enabled":false,"stepStatus":{}},"imageAssessment":{"enabled":false,"stepStatus":{}},"adaptableContentAssessment":{"enabled":false,"stepStatus":{}},"linksAssessment":{"enabled":false,"stepStatus":{}},"nativeWidgetsAssessment":{"enabled":false,"stepStatus":{}},"customWidgetsAssessment":{"enabled":false,"stepStatus":{}},"automatedChecks":{"enabled":false,"stepStatus":{"area-alt":false,"aria-allowed-attr":false,"aria-command-name":false}},"sequenceAssessment":{"enabled":false,"stepStatus":{}},"semanticsAssessment":{"enabled":false,"stepStatus":{}},"pointerMotionAssessment":{"enabled":false,"stepStatus":{}},"contrastAssessment":{"enabled":false,"stepStatus":{}}}},"scanning":null,"selectedFastPassDetailsView":1,"selectedAdhocDetailsView":1,"selectedDetailsViewPivot":2,"injectingStarted":false,"injectingRequested":false,"focusedTarget":null},"tabId":161} with browser error message: Cannot read properties of null (reading 'stop') sendMessageToTab(161) failed for message {"isStoreUpdateMessage":true,"storeId":"VisualizationStore","messageType":"STORE_CHANGED","storeType":1,"payload":{"tests":{"adhoc":{"headings":{"enabled":false},"issues":{"enabled":false},"landmarks":{"enabled":false},"tabStops":{"enabled":false},"color":{"enabled":false},"needsReview":{"enabled":false}},"assessments":{"headingsAssessment":{"enabled":false,"stepStatus":{}},"colorSensoryAssessment":{"enabled":false,"stepStatus":{}},"languageAssessment":{"enabled":false,"stepStatus":{}},"landmarksAssessment":{"enabled":false,"stepStatus":{}},"pageAssessment":{"enabled":false,"stepStatus":{}},"repetitiveContentAssessment":{"enabled":false,"stepStatus":{}},"keyboardInteractionAssessment":{"enabled":true,"stepStatus":{"keyboard-navigation":false,"no-keyboard-traps":true}},"audioVideoOnlyAssessment":{"enabled":false,"stepStatus":{}},"errorsAssessment":{"enabled":false,"stepStatus":{}},"timedEventsAssessment":{"enabled":false,"stepStatus":{}},"parsingAssessment":{"enabled":false,"stepStatus":{}},"prerecordedMultimediaAssessment":{"enabled":false,"stepStatus":{}},"liveMultimediaAssessment":{"enabled":false,"stepStatus":{}},"visibleFocusOrderAssessment":{"enabled":false,"stepStatus":{}},"imageAssessment":{"enabled":false,"stepStatus":{}},"adaptableContentAssessment":{"enabled":false,"stepStatus":{}},"linksAssessment":{"enabled":false,"stepStatus":{}},"nativeWidgetsAssessment":{"enabled":false,"stepStatus":{}},"customWidgetsAssessment":{"enabled":false,"stepStatus":{}},"automatedChecks":{"enabled":false,"stepStatus":{"area-alt":false,"aria-allowed-attr":false,"aria-command-name":false}},"sequenceAssessment":{"enabled":false,"stepStatus":{}},"semanticsAssessment":{"enabled":false,"stepStatus":{}},"pointerMotionAssessment":{"enabled":false,"stepStatus":{}},"contrastAssessment":{"enabled":false,"stepStatus":{}}}},"scanning":"no-keyboard-traps","selectedFastPassDetailsView":1,"selectedAdhocDetailsView":1,"selectedDetailsViewPivot":2,"injectingStarted":false,"injectingRequested":true,"focusedTarget":null},"tabId":161} with browser error message: Cannot read properties of null (reading 'stop') sendMessageToTab(161) failed for message {"isStoreUpdateMessage":true,"storeId":"VisualizationStore","messageType":"STORE_CHANGED","storeType":1,"payload":{"tests":{"adhoc":{"headings":{"enabled":false},"issues":{"enabled":false},"landmarks":{"enabled":false},"tabStops":{"enabled":false},"color":{"enabled":false},"needsReview":{"enabled":false}},"assessments":{"headingsAssessment":{"enabled":false,"stepStatus":{}},"colorSensoryAssessment":{"enabled":false,"stepStatus":{}},"languageAssessment":{"enabled":false,"stepStatus":{}},"landmarksAssessment":{"enabled":false,"stepStatus":{}},"pageAssessment":{"enabled":false,"stepStatus":{}},"repetitiveContentAssessment":{"enabled":false,"stepStatus":{}},"keyboardInteractionAssessment":{"enabled":true,"stepStatus":{"keyboard-navigation":false,"no-keyboard-traps":true}},"audioVideoOnlyAssessment":{"enabled":false,"stepStatus":{}},"errorsAssessment":{"enabled":false,"stepStatus":{}},"timedEventsAssessment":{"enabled":false,"stepStatus":{}},"parsingAssessment":{"enabled":false,"stepStatus":{}},"prerecordedMultimediaAssessment":{"enabled":false,"stepStatus":{}},"liveMultimediaAssessment":{"enabled":false,"stepStatus":{}},"visibleFocusOrderAssessment":{"enabled":false,"stepStatus":{}},"imageAssessment":{"enabled":false,"stepStatus":{}},"adaptableContentAssessment":{"enabled":false,"stepStatus":{}},"linksAssessment":{"enabled":false,"stepStatus":{}},"nativeWidgetsAssessment":{"enabled":false,"stepStatus":{}},"customWidgetsAssessment":{"enabled":false,"stepStatus":{}},"automatedChecks":{"enabled":false,"stepStatus":{"area-alt":false,"aria-allowed-attr":false,"aria-command-name":false}},"sequenceAssessment":{"enabled":false,"stepStatus":{}},"semanticsAssessment":{"enabled":false,"stepStatus":{}},"pointerMotionAssessment":{"enabled":false,"stepStatus":{}},"contrastAssessment":{"enabled":false,"stepStatus":{}}}},"scanning":"no-keyboard-traps","selectedFastPassDetailsView":1,"selectedAdhocDetailsView":1,"selectedDetailsViewPivot":2,"injectingStarted":false,"injectingRequested":false,"focusedTarget":null},"tabId":161} with browser error message: Cannot read properties of null (reading 'stop') ```

I think the most interesting parts the logs are:

1x sendMessageToTab(161) failed for message /* a normal looking AssessmentStore STORE_CHANGED message */ with browser error message: Cannot read properties of undefined (reading 'scanning')
3x sendMessageToTab(161) failed for message /* a normal looking VisualizationStore STORE_CHANGED message */ with browser error message: Cannot read properties of undefined (reading 'stop')

These error messages come from browser-message-broadcaster-factory.ts's errorHandler around the browser.tabs.sendMessage webextension API - the Cannot read properties of undefined (reading 'whatever') part of the error comes directly from the browser error given to us from that API.

The messages are very interesting because they imply that the browser is trying to read properties of the message which don't actually exist on the message in question. Particularly, the error with the AssessmentStore message talks about reading a scanning property, but there is no scanning property anywhere in AssessmentStoreData - there is in VisualizationStoreData, though.

I am suspicious that there is some browser regression related to interleaving these messages; we can interleave them in practice, since we end up fire-and-forgetting the asynchronous messages from the browser-message-broadcaster-factory's broadcast implementations. I found a few possibly-related Chromium items with recent activity, but none that are obviously the root cause here:

KJohansonMicrosoft commented 2 years ago

I was able to reproduce both this issue and the one you emailed about (text included below) by switching quickly between different assisted requirements. Thank you for pointing out this issue, we will work to fix it!

Thanks!

aut0poietic commented 2 years ago

Not sure if it helps, but I ran into the same issue and can consistently trigger this issue by enabling either the Keyboard > Keyboard navigation, Focus > Visible focus, or "Focus > Focus order" Visual helpers, then switching to an assessment item that uses a scan (Landmarks > Landmark roles, Page navigation > Page title).

It didn't seem to matter if I disabled the helper afterward or not. Weirdly, some previously performed scans would remain available, but I haven't narrowed down any logic behind that behavior.

Only way to resolve the issue is to close Accessibility Insights window, refresh the tab being assessed, then reopen Accessibility Insights.

ferBonnin commented 2 years ago

thanks for sharing this @aut0poietic!

Informational mostly for who looks into this bug, with these steps, I can repo this somewhat consistently in the production version of the extension, however I cannot repo in the canary version at all.

ferBonnin commented 2 years ago

Per conversation with Peter, let's look at this in Canary and try to repro (timebox this), if it doesn't repo, its likely we fixed it as a side effect of other work

Nimbobs95 commented 1 year ago

Hi, I am also having similar issues. When running the fast pass tool on most URLs it completes within ~5s but when running it on the page I am currently updating the tool gets stuck in the scanning state. I have left it running for ~30minutes and never yet seen it complete / offer any more info. I have sent an email to your internal email address with more info.

ferBonnin commented 1 year ago

@Nimbobs95 thank you for sharing this and working with our team to debug the issue! We confirmed the issue is resolved in our canary extension which will be released into production in the upcoming weeks.

DaveTryon commented 1 year ago

The release with the fix shipped in March. Closing.