GoogleChromeLabs / chromium-bidi

Implementation of WebDriver BiDi for Chromium
https://googlechromelabs.github.io/chromium-bidi/
Apache License 2.0
85 stars 28 forks source link

`test_input_performActions* ` e2e test is flaky #1407

Closed sadym-chromium closed 1 month ago

sadym-chromium commented 9 months ago

e2e tests of input are flaky. Example:

test_input_performActionsEmitsDragging: https://github.com/GoogleChromeLabs/chromium-bidi/actions/runs/6448243919/job/17509797173?pr=1403

Although the output is really hard to read, the difference is expected and missing the following part at the beginning:

      {
        type: 'object',
        value: [
          [
            'event',
            {
              type: 'string',
              value: 'mousemove',
            },
          ],
          [
            'button',
            {
              type: 'number',
              value: 0,
            },
          ],
          [
            'buttons',
            {
              type: 'number',
              value: 0,
            },
          ],
          [
            'clientX',
            {
              type: 'number',
              value: 630,
            },
          ],
          [
            'clientY',
            {
              type: 'number',
              value: 336,
            },
          ],
        ],
      },

Another example:

test_input_performActionsEmitsPointerEvents: Example: https://github.com/GoogleChromeLabs/chromium-bidi/actions/runs/6548857552/job/17784481994?pr=1443 Related: https://github.com/GoogleChromeLabs/chromium-bidi/issues/1407

jrandolf commented 9 months ago

This is a result of headful mode. The solution is to disable the test in headful.

sadym-chromium commented 9 months ago

This is a result of headful mode. The solution is to disable the test in headful.

Cannot we make the test more flexible, so that it tolerates the fluctuations?

sadym-chromium commented 8 months ago

friendly ping. This causes false-negative CI runs almost daily

sadym-chromium commented 8 months ago

another example: https://github.com/GoogleChromeLabs/chromium-bidi/actions/runs/6509030633/job/17679618738

sadym-chromium commented 1 month ago

The issue addressed, expectations do not tolerate flakiness anymore