element-hq / element-web

A glossy Matrix collaboration client for the web.
https://element.io
Apache License 2.0
10.96k stars 1.95k forks source link

[Disabled] Flaky playwright test: `Timeline › message displaying > should set inline start padding to a hidden event line` #26890

Open richvdh opened 7 months ago

richvdh commented 7 months ago

https://github.com/matrix-org/matrix-react-sdk/actions/runs/7559429855:

 1) [Rust Crypto] › timeline/timeline.spec.ts:555:13 › Timeline › message displaying › should set inline start padding to a hidden event line 
    Error: Screenshot comparison failed:

      28 pixels (ratio 0.01 of all image pixels) are different.

    Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/timeline/timeline.spec.ts/hidden-event-line-zero-padding-irc-layout-linux.png
    Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-displaying-should-set-inline-start-padding-to-a-hidden-event-line-Rust-Crypto/hidden-event-line-zero-padding-irc-layout-actual.png
        Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-displaying-should-set-inline-start-padding-to-a-hidden-event-line-Rust-Crypto/hidden-event-line-zero-padding-irc-layout-diff.png

    Call log:
      - page._expectScreenshot with timeout 5000ms
      -   verifying given screenshot expectation
      - waiting for locator('.mx_MainSplit')
      -   locator resolved to <div class="mx_MainSplit">…</div>
      - taking element screenshot
      -   disabled all CSS animations
      - waiting for fonts to load...
      - fonts loaded
      -   waiting for element to be visible and stable
      -   element is visible and stable
      - 28 pixels (ratio 0.01 of all image pixels) are different.
      - waiting 100ms before taking screenshot
      - waiting for locator('.mx_MainSplit')
      -   locator resolved to <div class="mx_MainSplit">…</div>
      - taking element screenshot
      -   disabled all CSS animations
      - waiting for fonts to load...
      - fonts loaded
      -   waiting for element to be visible and stable
      -   element is visible and stable
      - captured a stable screenshot
      - 28 pixels (ratio 0.01 of all image pixels) are different.

      585 |             };
      586 |
    > 587 |             await expect(page.locator(".mx_MainSplit")).toMatchScreenshot(
          |                                                         ^
      588 |                 "hidden-event-line-zero-padding-irc-layout.png",
      589 |                 screenshotOptions,
      590 |             );

        at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/timeline/timeline.spec.ts:587:57

Looking at the screenshot, the room avatar appears to be an "E" rather than a "T":

Diff:

image

image

t3chguy commented 7 months ago

2nd strike: https://github.com/element-hq/element-web/actions/runs/7639565834

t3chguy commented 7 months ago

This is https://github.com/element-hq/element-web/issues/22590

t3chguy commented 7 months ago

I'm not sure disabling this test will do much good, given the failure mode can happen to any test

MidhunSureshR commented 6 months ago

3rd strike: https://github.com/element-hq/element-web/actions/runs/7744210809

richvdh commented 6 months ago

@MidhunSureshR: I updated the summary.

github-actions[bot] commented 4 months ago

This flaky test issue has not been updated in 14 days. It is being closed as presumed resolved.

github-actions[bot] commented 3 months ago

This flaky test issue has not been updated in 14 days. It is being closed as presumed resolved.