[Table of Contents] e2e tests for scroll giving unstable results: DDS Consulting

carbon-design-system / carbon-for-ibm-dotcom

Carbon for IBM.com is based on the Carbon Design System for IBM

https://www.ibm.com/standards/carbon/

Apache License 2.0

268 stars 156 forks source link

[Table of Contents] e2e tests for scroll giving unstable results: DDS Consulting #7770

Closed jeffchew closed 2 years ago

jeffchew commented 2 years ago

Engineering info:

HC engineer: @andy-blum
HC JIRA Ticket: https://jsw.ibm.com/browse/HC-2332
DDS consulting engineer:

Description

The e2e test should update current section on scroll in table of contents is currently giving unstable results, which is failing the e2e test runs. This should be looked at to make the results more stable then reactivate.

Component(s) impacted

Table of Contents

Browser

Chrome

Carbon for IBM.com version

v1.28.0

Severity

Severity 4 = The problem is not visible to or noticeable to an average user. Affects minor functionality, no workaround needed.

Application/website

Carbon for IBM.com

Package

@carbon/ibmdotcom-react, @carbon/ibmdotcom-web-components

CodeSandbox example

n/a, see CI reports

Steps to reproduce the issue (if applicable)

No response

Release date (if applicable)

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I checked the current issues for duplicate issues

jeffchew commented 2 years ago

This was suppressed in this PR: https://github.com/carbon-design-system/carbon-for-ibm-dotcom/pull/7763

andy-blum commented 2 years ago

@jeffchew I've re-run the test several times locally, and not to be all it-works-on-my-machine, but it works on my machine. Any ideas as to why it might be failing in the pipelines?

jeffchew commented 2 years ago

Hey @andy-blum , perhaps if you can open up a draft PR and enable the tests again and see if it's a CI thing? Ci has lower resources so might be something there too.

Also, perhaps you can also check locally by switching to firefox to see if you're also seeing issues?

proeung commented 2 years ago

@jeffchew Is there anything that we can do to push this issue forward? I believe there was a discussion of doing an on the fly BrowserStack testing?

annawen1 commented 2 years ago

@jeffchew, looking at the browserstack automation dashboard as of today, it looks like all the tests for TOC are passing consistently with the exception of FF on Windows 10 for TOC web components. There aren't much details on why the job for FF Win 10 was terminated, just that there is a banner notification as shown below. This possibly has something to do with Browserstack/Cypress instead of the test code.

CC @andy-blum

annawen1 commented 2 years ago

I no longer see the original errors reported here for TOC coming up in the browserstack automation dashboard. Will close this ticket.