search

theintern / intern

A next-generation code testing stack for JavaScript.
https://theintern.io/
Other
4.36k stars 311 forks source link

Self tests don’t pass on Safari 13 #1060

Closed jason0x43 closed 4 years ago

jason0x43 commented 4 years ago

Both locally and on BS, a number of functional self tests are failing on Safari 13.

jason0x43 commented 4 years ago

One unit test in common/ErrorFormatter also fails on BS.

Partway through the functional tests on BS, individual tests will start to fail with java.net.ConnectionException: Connection refused. Several tests in a row will fail this way, but others still pass. A typical run:

‣ Created remote session Safari 13 on macOS (4be49fb1bfdec7bdd02a60c12265785fa80eeedf)
...........................................................................................................................................
× Safari 13 on macOS - core/lib/common/ErrorFormatter - #format - error (0.001s)
    AssertionError: expected 'Error: foo\nerror\n\n\n  at promiseReactionJob @ [native code]' to match /^Error: foo\n\s{2}at (?:\S*\.)?error /
    AssertionError

    assertMatch
    methodWrapper
    [native code]

    error

      at promiseReactionJob @ [native code]

...................................................................................................................................................................................................
× Safari 13 on macOS - functional/webdriver/Command - #moveMouseTo usesElement (1.261s)
    UnknownError: [POST https://(redacted)@hub.browserstack.com:443/wd/hub/session/4be49fb1bfdec7bdd02a60c12265785fa80eeedf/buttonup / {}] java.net.ConnectException: Connection refused
      at Server.post @ src/webdriver/Server.ts:391:17
      at runRequest @ src/webdriver/Session.ts:132:48
      at common_1.Task.cancelled @ src/webdriver/Session.ts:169:45
      at new Task @ src/common/lib/Task.ts:327:7
      at Session._delegateToServer @ src/webdriver/Session.ts:110:12
      at Session.serverPost @ src/webdriver/Session.ts:181:17
      at Session.releaseMouseButton @ src/webdriver/Session.ts:1647:17
      at sessionMethod @ src/webdriver/Command.ts:839:45
      at Command.<anonymous> @ src/webdriver/Command.ts:857:18
      @ src/common/lib/Task.ts:456:23
      at Command._callSessionMethod @ src/webdriver/Command.ts:824:12
      at Command.releaseMouseButton @ src/webdriver/Command.ts:1386:17
      at Test.#moveMouseTo usesElement [as test] @ tests/functional/webdriver/Command.ts:294:12
      @ src/core/lib/Test.ts:267:52
      @ src/common/lib/Task.ts:456:23
      at runMicrotasks @ anonymous
      at processTicksAndRejections @ internal/process/task_queues.js:97:5

Suite Safari 13 on macOS - functional/webdriver/Command ERROR in beforeEach
UnknownError: [POST https://(redacted)@hub.browserstack.com:443/wd/hub/session/4be49fb1bfdec7bdd02a60c12265785fa80eeedf/url / {"url":"about:blank"}] java.net.ConnectException: Connection refused
  at Server.post @ src/webdriver/Server.ts:391:17
  at runRequest @ src/webdriver/Session.ts:132:48
  at common_1.Task.cancelled @ src/webdriver/Session.ts:169:45
  at new Task @ src/common/lib/Task.ts:327:7
  at Session._delegateToServer @ src/webdriver/Session.ts:110:12
  at Session.serverPost @ src/webdriver/Session.ts:181:17
  at Session.get @ src/webdriver/Session.ts:361:17
  at Session.session.get @ tests/functional/webdriver/support/util.ts:34:21
  at Suite.beforeEach @ tests/functional/webdriver/Command.ts:23:22
  @ src/core/lib/Suite.ts:431:54
Suite Safari 13 on macOS - functional/webdriver/Element ERROR in before
UnknownError: [POST https://(redacted)@hub.browserstack.com:443/wd/hub/session/4be49fb1bfdec7bdd02a60c12265785fa80eeedf/timeouts / {"type":"implicit","ms":5000}] java.net.ConnectException: Connection refused
  at Server.post @ src/webdriver/Server.ts:391:17
  at runRequest @ src/webdriver/Session.ts:132:48
  at common_1.Task.cancelled @ src/webdriver/Session.ts:169:45
  at new Task @ src/common/lib/Task.ts:327:7
  at Session._delegateToServer @ src/webdriver/Session.ts:110:12
  at Session.serverPost @ src/webdriver/Session.ts:181:17
  at Session.setTimeout @ src/webdriver/Session.ts:239:26
  at Session.setFindTimeout @ src/webdriver/Session.ts:2151:17
  @ tests/functional/webdriver/support/util.ts:48:29
  @ src/common/lib/Task.ts:456:23
Suite Safari 13 on macOS - functional/webdriver/helpers/pollUntil ERROR in before
UnknownError: [POST https://(redacted)@hub.browserstack.com:443/wd/hub/session/4be49fb1bfdec7bdd02a60c12265785fa80eeedf/timeouts / {"type":"implicit","ms":5000}] java.net.ConnectException: Connection refused
  at Server.post @ src/webdriver/Server.ts:391:17
  at runRequest @ src/webdriver/Session.ts:132:48
  at common_1.Task.cancelled @ src/webdriver/Session.ts:169:45
  at new Task @ src/common/lib/Task.ts:327:7
  at Session._delegateToServer @ src/webdriver/Session.ts:110:12
  at Session.serverPost @ src/webdriver/Session.ts:181:17
  at Session.setTimeout @ src/webdriver/Session.ts:239:26
  at Session.setFindTimeout @ src/webdriver/Session.ts:2151:17
  @ tests/functional/webdriver/support/util.ts:48:29
  @ src/common/lib/Task.ts:456:23
Suite Safari 13 on macOS - functional/webdriver/helpers/pollUntilTruthy ERROR in before
UnknownError: [POST https://(redacted)@hub.browserstack.com:443/wd/hub/session/4be49fb1bfdec7bdd02a60c12265785fa80eeedf/timeouts / {"type":"implicit","ms":5000}] java.net.ConnectException: Connection refused
  at Server.post @ src/webdriver/Server.ts:391:17
  at runRequest @ src/webdriver/Session.ts:132:48
  at common_1.Task.cancelled @ src/webdriver/Session.ts:169:45
  at new Task @ src/common/lib/Task.ts:327:7
  at Session._delegateToServer @ src/webdriver/Session.ts:110:12
  at Session.serverPost @ src/webdriver/Session.ts:181:17
  at Session.setTimeout @ src/webdriver/Session.ts:239:26
  at Session.setFindTimeout @ src/webdriver/Session.ts:2151:17
  @ tests/functional/webdriver/support/util.ts:48:29
  @ src/common/lib/Task.ts:456:23
........
~ Safari 13 on macOS - functional/webdriver/Server - #getSessionCapabilities (Disabled until support for W3C mode is implemented)
..
Suite Safari 13 on macOS - functional/webdriver/Session ERROR in before
Error: connect ETIMEDOUT 207.254.8.4:443
  at Server.post @ src/webdriver/Server.ts:391:17
  at runRequest @ src/webdriver/Session.ts:132:48
  at common_1.Task.cancelled @ src/webdriver/Session.ts:169:45
  at new Task @ src/common/lib/Task.ts:327:7
  at Session._delegateToServer @ src/webdriver/Session.ts:110:12
  at Session.serverPost @ src/webdriver/Session.ts:181:17
  at Session.setTimeout @ src/webdriver/Session.ts:239:26
  at Session.setFindTimeout @ src/webdriver/Session.ts:2151:17
  @ tests/functional/webdriver/support/util.ts:48:29
  @ src/common/lib/Task.ts:456:23
TOTAL: tested 1 platforms, 344 passed, 2 failed, 1 skipped, 229 not run; suite error occurred
jason0x43 commented 4 years ago

The failing unit test was happening because Safari 13 stack traces are weird. Stack lines may not have sources, and empty lines are included for anonymous functions.

Screen Shot 2020-01-23 at 1 59 13 PM
jason0x43 commented 4 years ago

This is resolved (at least when running Safari 13 locally) as of d319d9f.