elixir-wallaby / wallaby

Concurrent browser tests for your Elixir web apps.
https://twitter.com/elixir_wallaby
MIT License
1.68k stars 198 forks source link

Improve crash decoding malformed JSON from Chromedriver #778

Closed s3cur3 closed 5 months ago

s3cur3 commented 5 months ago

Hi Mitch! When upgrading from 0.30.6 to 0.30.7, I started seeing a crash in Wallaby.HTTPClient.check_for_response_errors/1 during JSON decode when an unexpected alert is opened. The message coming back from Chromedriver looks like this:

%{
  "sessionId" => "ff16e3040b4a88c33e77c4af572cddd9",
  "status" => 26,
  "value" => %{
    "message" => "unexpected alert open: {Alert text : Error loading data, please refresh and try again}\n  (Session info: chrome-headless-shell=125.0.6422.142)\n  (Driver info: chromedriver=125.0.6422.78 (14db42ec38aded3304a3e624a0a038e02956b87e-refs/branch-heads/6422@{#1088}),platform=Mac OS X 14.5.0 x86_64)"
  }
}

The error in the test runner looks like this:

1) feature blah blah blah
     test/my_app_web/e2e/my_test.exs:90
     ** (Jason.DecodeError) unexpected byte at position 1: 0x41 ("A")
     code: |> click(Query.button("Log in"))
     stacktrace:
       (jason 1.4.1) lib/jason.ex:92: Jason.decode!/2
       (wallaby 0.30.7) lib/wallaby/httpclient.ex:171: Wallaby.HTTPClient.coerce_json_message/1
       (wallaby 0.30.7) lib/wallaby/httpclient.ex:106: Wallaby.HTTPClient.check_for_response_errors/1
       (wallaby 0.30.7) lib/wallaby/httpclient.ex:56: Wallaby.HTTPClient.make_request/5
       (wallaby 0.30.7) lib/wallaby/webdriver_client.ex:45: Wallaby.WebdriverClient.find_elements/2
       (wallaby 0.30.7) lib/wallaby/driver/log_checker.ex:6: Wallaby.Driver.LogChecker.check_logs!/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:1501: anonymous fn/3 in Wallaby.Browser.execute_query/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:148: Wallaby.Browser.retry/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:951: Wallaby.Browser.find/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:996: Wallaby.Browser.find/3
       test/my_app_web/e2e/my_test.exs:101: (test)

By falling back to the not-quite-JSON message like this, I still end up getting a crash, but the error message is at least in the error message:

  2) feature blah blah blah
     test/my_app_web/e2e/my_test.exs:44
     ** (FunctionClauseError) no function clause matching in Wallaby.WebdriverClient.cast_as_element/2

     The following arguments were given to Wallaby.WebdriverClient.cast_as_element/2:

         # 1
         %Wallaby.Session{id: "13bcabde322a27b712ffdd2539b35b85", url: "http://localhost:64949/session/13bcabde322a27b712ffdd2539b35b85", session_url: "http://localhost:64949/session/13bcabde322a27b712ffdd2539b35b85", driver: Wallaby.Chrome, capabilities: %{version: "", unhandledPromptBehavior: "accept", chromeOptions: %{args: ["--no-sandbox", "window-size=1280,800", "--disable-gpu", "--headless", "--fullscreen", "--user-agent=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36/BeamMetadata (g2gCdwJ2MXQAAAADdwVvd25lclh3DW5vbm9kZUBub2hvc3QAAAVEAAAAAAAAAAB3CXRyYXBfZXhpdHcEdHJ1ZXcEcmVwb2wAAAABdxZFbGl4aXIuRmVsdFNlcnZlci5SZXBvag==)"], binary: "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"}, cssSelectorsEnabled: true, javascriptEnabled: false, loadImages: false, loggingPrefs: %{browser: "DEBUG"}, nativeEvents: false, platform: "ANY", rotatable: false, takesScreenshot: true}, server: Wallaby.Chrome.Chromedriver, screenshots: []}

         # 2
         {"message", "{Alert text : Error loading data, please refresh and try again}"}

     Attempted function clauses (showing 2 out of 2):

         defp cast_as_element(parent, %{"ELEMENT" => id})
         defp cast_as_element(parent, %{"element-6066-11e4-a52e-4f735466cecf" => id})

     code: |> click(Query.button("Share"))
     stacktrace:
       (wallaby 0.30.7) lib/wallaby/webdriver_client.ex:681: Wallaby.WebdriverClient.cast_as_element/2
       (elixir 1.16.2) lib/enum.ex:1708: anonymous fn/3 in Enum.map/2
       (stdlib 5.2) maps.erl:416: :maps.fold_1/4
       (elixir 1.16.2) lib/enum.ex:2540: Enum.map/2
       (wallaby 0.30.7) lib/wallaby/webdriver_client.ex:47: Wallaby.WebdriverClient.find_elements/2
       (wallaby 0.30.7) lib/wallaby/driver/log_checker.ex:6: Wallaby.Driver.LogChecker.check_logs!/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:1501: anonymous fn/3 in Wallaby.Browser.execute_query/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:148: Wallaby.Browser.retry/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:951: Wallaby.Browser.find/2
       (wallaby 0.30.7) lib/wallaby/browser.ex:996: Wallaby.Browser.find/3
       test/my_app_web/e2e/my_test.exs:63: (test)

For what it's worth, I'm on Chromedriver 125.0.6422.141 + Chrome 125.0.6422.142.

mhanberg commented 5 months ago

Thanks, I haven't really been able to find any documentation on these seemingly new errors.

To clarify, did your app actually pop open an alert with text "Error loading data, please refresh and try again" or is that some unknown text?

s3cur3 commented 5 months ago

The app did indeed pop open that alert! I actually just realized there's a better fix (one that gets the unhandled alert actually ignored, as the default capabilities settings specify)... #779 incoming. 🤦‍♂️