Invalid JSON being recieved by browerless/chrome

passbe commented 1 year ago

Step To Reproduce / Observed behavior

I've spent a while trying to get to the bottom of this but have exhausted my knowledge here. I have the following docker container running (from my docker-compose.yml file):

browser:
    image: browserless/chrome
    restart: unless-stopped
    environment:
      - DEBUG=*
      - ENABLE_DEBUGGER=true
      - PREBOOT_CHROME=true

By using the documented code snippet I run the following command:

Playwright.connect_to_playwright_server('ws://browser:3000') do |playwright|
  playwright.chromium.launch do |browser|
    page = browser.new_page
    page.goto('https://github.com/YusukeIwaki')
    page.screenshot(path: './YusukeIwaki.png')
  end
end

Which throws the following error:

#<Thread:0x00007fe38023d000 /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_client.rb:101 run> terminated with exception (report_on_exception is true):
/usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/errors.rb:5:in `parse': undefined method `[]' for nil:NilClass (NoMethodError)

      if error_payload['name'] == 'TimeoutError'
                      ^^^^^^^^
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/connection.rb:143:in `dispatch'
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/connection.rb:11:in `block in initialize'
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_transport.rb:93:in `handle_on_message'
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_transport.rb:74:in `block in async_run'
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_client.rb:161:in `handle_on_message'
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_client.rb:90:in `block in setup'
        from /usr/local/bundle/gems/websocket-driver-0.7.5/lib/websocket/driver/event_emitter.rb:39:in `block in emit'
        from /usr/local/bundle/gems/websocket-driver-0.7.5/lib/websocket/driver/event_emitter.rb:38:in `each'
        from /usr/local/bundle/gems/websocket-driver-0.7.5/lib/websocket/driver/event_emitter.rb:38:in `emit'
        from /usr/local/bundle/gems/websocket-driver-0.7.5/lib/websocket/driver/hybi.rb:408:in `emit_message'
        from /usr/local/bundle/gems/websocket-driver-0.7.5/lib/websocket/driver/hybi.rb:390:in `emit_frame'
        from /usr/local/bundle/gems/websocket-driver-0.7.5/lib/websocket/driver/hybi.rb:118:in `parse'
        from /usr/local/bundle/gems/websocket-driver-0.7.5/lib/websocket/driver/client.rb:63:in `parse'
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_client.rb:95:in `wait_for_data'
        from /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_client.rb:102:in `block in start'

The communication is received by the browser docker container as it emits the following messages:

2023-05-01T13:03:59.284Z browserless:job DNUFVB3LDB41LFLPC1655Z2RU8FAATZA: /: Inbound WebSocket request.                                                      
2023-05-01T13:03:59.291Z browserless:hardware Checking overload status: CPU 6% Memory 43%                                                                     
2023-05-01T13:03:59.292Z browserless:job DNUFVB3LDB41LFLPC1655Z2RU8FAATZA: Adding new job to queue.                                                           
2023-05-01T13:03:59.293Z browserless:server Starting new job                                                                                                  
2023-05-01T13:03:59.293Z browserless:system Waiting pre-booted chrome instance                                                                                
2023-05-01T13:03:59.293Z browserless:job DNUFVB3LDB41LFLPC1655Z2RU8FAATZA: Getting browser.                                                                                                                                                
2023-05-01T13:03:59.294Z browserless:system Got chrome instance                                                                                               
2023-05-01T13:03:59.294Z browserless:job DNUFVB3LDB41LFLPC1655Z2RU8FAATZA: Starting session.                                                                  
2023-05-01T13:03:59.294Z browserless:job DNUFVB3LDB41LFLPC1655Z2RU8FAATZA: Proxying request to /devtools/browser route: ws://127.0.0.1:34393/devtools/browser/8251171f-21c8-4194-b0b7-4188ed78eaa2.                                         
2023-05-01T13:03:59.297Z puppeteer:protocol:RECV ◀ [                                                                                                          
  '{"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"30f0f222-cdaa-40e2-9499-0816088ecea7","type":"browser","title":"","url":"","attached":false,"canAccessOpener":false}}}'                                             
]                                                                                                                                                             
2023-05-01T13:03:59.297Z puppeteer:protocol:RECV ◀ [                                                                                                          
  '{"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"30f0f222-cdaa-40e2-9499-0816088ecea7","type":"browser","title":"","url":"","attached":true,"canAccessOpener":false}}}'                                          
]

Debugging the issue further up the call stack I can see the following response from the docker container:

Stop by #1  BP - Line  /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_transport.rb:93 (line)
(rdbg) i    # info command
%self = #<Playwright::WebSocketTransport:0x00007faf4a39f9d0 @ws_endpoint="ws://browser:3000/ws", @de...>
data = "{\"id\":1,\"error\":{\"code\":-32600,\"message\":\"Message has property other than 'id', 'met...
obj = {"id"=>1, "error"=>{"code"=>-32600, "message"=>"Message has property other than 'id', 'method',...
@debug = false
@on_driver_crashed = #<Proc:0x00007faf4a7e98b0 /usr/local/bundle/gems/playwright-ruby-client-1.33.0/...>
@on_message = #<Proc:0x00007faf4a7ea3f0 /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/pla...>
@ws = #<Playwright::WebSocketClient:0x00007faf4a39f8e0 @impl=#<Playwright::WebSocketClient::DriverIm...>
@ws_endpoint = "ws://browser:3000/ws"
(rdbg) data
"{\"id\":1,\"error\":{\"code\":-32600,\"message\":\"Message has property other than 'id', 'method', 'sessionId', 'params'\"}}"
(rdbg)

So it appears we are sending malformed JSON. Setting a breakpoint at web_socket_transport.rb:27 I can see the message we are sending:

Stop by #0  BP - Line  /usr/local/bundle/gems/playwright-ruby-client-1.33.0/lib/playwright/web_socket_transport.rb:27 (line)
(rdbg) message
{:id=>1, :guid=>"", :method=>"initialize", :params=>{:sdkLanguage=>"python"}, :metadata=>{}}

Has something recently changed in the API which is causing the issue?

Expected behavior

To communicate with the websocket service correctly.

Environment

Ruby: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux-musl]
Rails: 7.0.4.2
playwright-ruby-client: 1.33.0
browserless/chrome: latest (Digest:sha256:11a7103f1d40ce3c04dbebdfff3f60fd01d9f1975763957b4557263eb5564c14)

Appreciate any help.

YusukeIwaki commented 1 year ago

Playwright is not supported on browserless/chrome. (ref: https://github.com/browserless/chrome/issues/1419 ) Probably, we can use playwright.chromium.connect_over_cdp instead of Playwright.connect_to_playwright_server.

passbe commented 1 year ago

Thank you for getting back to me so quickly. It appears I'm not able to call connect_over_cdp without a Playwright block already established which requires specifying playwright_cli_executable_path.

The documentation here shows it should be possible however, how do you establish a playwright instance to use?

YusukeIwaki commented 1 year ago

I'm afraid that I made a bit misunderstanding.

It seems that recent browserless/chrome uses playwright.chromium.launchServer internally. https://github.com/browserless/chrome/blob/7cff7da00b155034a142b5244f6fb81b6749c327/src/chrome-helper.ts#L595 So probably we can use Playwright.connect_to_browser_server instead of connect_to_playwright_server for connecting the browserless from playwright-ruby-client.

Could you try this?

Playwright.connect_to_browser_server('ws://browser:3000') do |browser|
    page = browser.new_page
    page.goto('https://github.com/YusukeIwaki')
    page.screenshot(path: './YusukeIwaki.png')
end

passbe commented 1 year ago

@YusukeIwaki this does seem to be the case and the code snippet does connect. Thank you for your help on this issue. :+1:

YusukeIwaki / playwright-ruby-client