iTwin / itwinjs-core

Monorepo for iTwin.js Library
https://www.itwinjs.org
MIT License
606 stars 210 forks source link

Flaky RenderTarget test #4420

Open pmconne opened 2 years ago

pmconne commented 2 years ago

Describe the bug The following test failure in core-full-stack-tests occasionally occurs in CI jobs.

1) RenderTarget
       should override symbology:

      AssertionError: expected 4 to equal 2
      + expected - actual

      -4
      +2

      at D:/vsts_a/279/s/full-stack-tests/core/src/frontend/standalone/RenderTarget.test.ts:368:32
      at testOnScreenViewport (D:/vsts_a/279/s/full-stack-tests/core/src/frontend/TestViewport.ts:336:5)
      at testViewports (D:/vsts_a/279/s/full-stack-tests/core/src/frontend/TestViewport.ts:360:3)
      at testViewportsWithDpr (D:/vsts_a/279/s/full-stack-tests/core/src/frontend/TestViewport.ts:368:5)
      at Context.<anonymous> (D:/vsts_a/279/s/full-stack-tests/core/src/frontend/standalone/RenderTarget.test.ts:264:5)

To Reproduce Submit PR. See if failure above occurs.

Expected behavior Test should pass consistently.

Desktop (please complete the applicable information):

Additional context Most recent occurrence was in #4414. Seems to have begun occurring maybe 1-2 months prior?

The specific "actual" value reported seems to differ from one failing run to the next.

pmconne commented 2 years ago

@markschlosseratbentley might this have coincided with your switching the tests to use waitForSceneCompletion?

markschlosseratbentley commented 2 years ago

@markschlosseratbentley might this have coincided with your switching the tests to use waitForSceneCompletion?

@pmconne A specific test -- WaitForSceneCompletion.test.ts -- was added to exercise the waitForSceneCompletion API. The rest of the tests all should still be using whatever their pre-existing wait logic entailed.

pmconne commented 2 years ago

Seems to only occur on Windows.

markschlosseratbentley commented 1 year ago

This is not reproducible so far locally. @markschlosseratbentley try on Windows.

markschlosseratbentley commented 1 year ago

Check if skipped. Consider priority then.

markschlosseratbentley commented 1 year ago

Test is skipped; moving to Proposed so time can be spent looking at higher priority flaky tests.